I want to export 4k videos faster, almost unlimited money, help me to build a warmachine

[loginv]

Hello,

First let me present the actual setup : At the moment, we have 3 different options for the gpu : 1x GTX 1080, 1x GTX 970 or 2x GTX 770 connected by SLI.
The processor is an i7 6700k.
The RAM is 32 go ddr4 2133 mhz.

We are editing 4k videos which we are exporting into h.264. The bitrate is 2,7 mb/s.
We are taking the videos from a NAS Synology DS416PLAY and we are exporting there.
We have 4 HDD in this NAS, ref. --> WD40EFRX-68WT0N0

The way we edit our videos is really simple : we insert an intro/outro on Premiere Pro CC 2017, we have to sync 2 different videos (a capture and a screencast), an overlay and we also have to apply an effect (which i don't remember the exact name) to remove the fish-eye effect (something like go pro lens correction). Also our videos are between 30 mins and 1h40.

We have 3 good computers for editing which takes the medias from the NAS, edit it and then export to the NAS, then we upload the exported videos on internet. So the connected thing and the possibility to share the base medias are something we would like to keep.

We almost have an unlimited budget and we want to build a huge setup (like... not 100.000€ but 5.000€ is fine i guess) only for the video export on a queue on Media Encoder (editing is fine) dedicated to the exporting of these videos. The computer would export videos 24/24h while we edit videos on the others computers. What should we change ? We searched a long time on the web without a real answer. Some say that SSD is the solution, others say than an i7 6950X is the way to go, ... We can change everything in the setup, can you help us getting the fabulous setup we want ?

Thank you very much.

[Andrew_S]

My comment is about the fish-eye problem. The premiere filter to correct for this takes a long time to process the video. This will be slowing down your exporting time. The best way is to use GoPro studio (get it from GoPro web site) and pre-process your GoPro footage with it to remove the fish eye, before putting it into Premiere. GoPro studio does the processing much quicker and trims off less of the picture than Premiere.

[loginv]

Thank you very much, this is going to help us a lot. I didn't know about that (we aren't using GoPros but we are facing a smiliar problem) and this solved like 70% of the problem.
I am still looking for a good setup/upgrade about the 4k export, so I won't tag the thread as "resolved" but thanks a lot !

[Ann Bens]

The fish eye or Lens Distortion Removal effect is now a GPU accelerated effects. (11.0.1.)

[moving thread to Hardware forum for specialized help]

[loginv]

Oh okay, thank you for the answer and for moving it, I'm sorry I never posted on these forums so I'm not very used to it.

[JFPhoton]

There are in depth articles on the Puget Systems website , where various combinations of CPUs,GPUs, and storage devices have been tested for performance tasks which include exporting speed.

Additionally, Bill Gehrke's PPBM 7 benchmark test for PPro has extensive test results from many different editing machines featuring differing components.

All in all, your current CPU,GPU,memory components are in a " usable" state for your tasks,but, are not in the middle or top tier category for performance. To get best performance with what you have , you may need to :

1. Use the benefit provided by the "K" feature of your CPU and safely overclock that chip to over 4 Ghz.......4.3 to 4.5 Ghz is possible and will provide a performance increase.

2. If you do have the 1080 GPU.....USE IT !! The 2x 770 in SLI is useless with PPro which does NOT use " SLI". However, PPro DOES use multiple GPUs in certain tasks, as Bill has found and tested. The single 1080 would outperform all the other GPUs you have and might even have to "wait" on your CPU and storage to feed it.

You can gain performance on the 1080 by using certain free utilities....like MSI Afterburner....to safely increase the " memory clock" setting of the GPU. This will help increase the speed of processing anything on the timeline which is " GPU accelerated" during export.

3. As documented in the Puget systems articles and by Bill's testing, the speed of your storage devices is MOST IMPORTANT concerning the good performance in editing, and the speed of exporting. Since you have all your media on a remote NAS and are also exporting to this, unless there is a very high data transfer speed between the computer and the NAS, there will be a large "bottleneck" and massive hindrance of performance. The speed of the NAS should be tested .

    Puget Systems has found that for best performance, only the OS, programs, and Windows page file should be on your "boot drive", ( preferably a quality Samsung 850Pro SSD ). The media cache and cache files must NOT be on your boot drive, according to them. Simply adding a new Samsung 960 Pro PCI SSD in the computer to act as the " project drive" will maximize performance during editing and exporting. Copy any media involved in the editing to it and export to it. Then , the rendered export files on that PCI SSD can simply be copied ,or, transferred wherever they need to go.

I have just looked at the "Tweaktown" review of your NAS and from what I see there, the performance is ABYSMAL for use in video editing...especially 4K !

Furthermore, the individual WD Red drives are " variable speed", which is horrible for use in video work......top claimed speed is 146 MB/sec. for each. The connection of the NAS are USB 3........this unit will serve as a slow as molasses backup or archive only......not fit for use in editing,or, for exporting to.

[RoninEdits]

JFPhoton wrote:

Puget Systems has found that for best performance, only the OS, programs, and Windows page file should be on your "boot drive"... The media cache and cache files must NOT be on your boot drive, according to them...

that is incorrect. their tests show about 0.5%-6%+ difference in favor of placing cache on the os/apps drive. which makes sense as its mostly idle. this also matches bills testing that cache on the os/apps drive had no penalty on performance.

https://www.pugetsystems.com/labs/articles/Adobe-Premiere-Pro-CC-2015-4-Storage-Optimization-854/

the first chart (media import 1&2 drives) has very strange results and must be in error, so it should be disregarded. its overlooked by the person that wrote the article and therefore the recommendations at the articles end are also in error with that data.

[JFPhoton]

I am aware that Bill has found in his testing that there " is no performance difference" in placing the media cache and cache files on the boot drive, where they can be easily erased from when a project is complete. However, Puget claims in this recent article that there IS a difference when importing footage .....as much as over 600%....concerning THAT one operation of "importing files". They REPEATED this claim again in their conclusion.  It may be worth testing yourself to verify if they have made a mistake concerning the " importing files" performance.

Eric at ADK computer systems can offer you specific advice for your particular situation...he would be worth looking up and contacting as he would know exactly what may be your best solution for what you are trying to do.

[RoninEdits]

JFPhoton wrote:

Puget claims in this recent article that there IS a difference when importing footage .....as much as over 600%....concerning THAT one operation of "importing files". They REPEATED this claim again in their conclusion.

that's exactly what i pointed out in my previous post, as the incorrect data in the first chart that lead to an incorrect conclusion. when disregarding that incorrect test result and looking at the rest of the charts, they show a benefit from having the cache on the os/apps drive when using two sata ssd's. if a fast m.2 drive is being used for the media drive or an everything drive, then i would agree to place cache on it if there is space, and the puget results show a performance benefit to that.

[RoninEdits]

for the AME render machine the cpu(s) is(are) important, but the rest of the hardware must be balanced or they will be a bottleneck. there are lots of options from an i7 8-10 core, a single or dual xeon system, or used older dual xeon systems. these machines could be built from parts or ordered complete, or mostly complete and finished by adding memory, storage, and gpu's etc. if you have only a few editors, an 8 or 10 core i7 might be able to keep up with render jobs from the editors and would make the build simpler. the more cpu cores in use, the more it will need faster storage, more memory, and more gpu's...

another problem might be AME. its built partially from premiere which only uses 6-8 cpu cores on average while encoding/exporting. so if you were to build a dual xeon machine with 20+ cpu cores, you might have to figure out how to get AME to render multiple jobs simultaneously. perhaps by using multiple instances of AME, perhaps with multiple virtual machines...

if the render machine is reading the media from the nas, the nas and network have to be able to keep up with the editing machines and the render machine or the nas/network will be a bottleneck. if that is a problem there are several solutions, like mirroring/syncing the files on the nas to a local drive inside the render machine, or even turning the render machine into the nas/file server. if the nas is limited to 1gbe, using the render machine for the file server could also open options for 10gbe networking.