for After Effects advice, Eric on the forum here can give you the best info. However, I have seen him say many times that AE is MEMORY INTENSIVE,....the more, the better. You HAVE 64GB...that seems very adequate for most workflows.
The OTHER thing he says is that other than Ray-Tracing, the GPU is HARDLY used at all.....CPU speed and number of cores are a BIG factor in AE performance.....your 3930k overclocked should be adequate....the MORE OC, the BETTER.
Next is the drive system....you have NOT described EXACTLY what you are doing NOW....you mentioned what your PLAN is,but for , right NOW you said : " I run/keep my project files on yet another SSD and have other drives holing media. "
HERE is where you MAY be doing something that may....or may NOT.....be causing slowdowns. With AE it is RECOMMENDED to use a dedicated drive for the "Global Performance Cache"...the HIGHER the speed of this drive, the BETTER, as AE STORES already "rendered" frames, ( for "previews", of course), on it so it DOESN"T have to RE-RENDER ALL the frames all over again every time you make a change to your work !!! So, in YOUR case : your OS drive is great...Samsung Pro models are the BEST...you have the 840 Pro ,which is only SLIGHTLY behind the 850 Pro in performance. ANOTHER 840,or, 850 Pro would be GREAT as the " Global performance cache drive" AND would also be the place to put ALL previews,Cache files, media cache files for Premier Pro,too. The overall speed and less latency of the SATAIII SSD will surpass the performance of a 2 disk RAID 0....even a 3 disk RAID 0.
Next....the speed at which you SERVE UP the video files can be important in preventing a slowdown.....ESPECIALLY if using large, uncompressed files. Here, you have options.....it is NOT necessary to put your project files on a SEPARATE drive from your media...you just want to make sure that the drive with your media and project files is as fast as the OTHERS in your system ! SO, you COULD use a THIRD SSD as a "working drive" for ACTIVE projects only.....a 1TB SSD perhaps...Crucial M550 is CHEAPER than the Samsung and as good as the 840 pros. As the CAPACITY of SSDs are small...you would MOVE all media and completed project files to HDDs for archiving and storage....the use of RAID with your existing HDDs would NOT be needed for this function.
Another option would be to take FOUR quality 1TB HDDs...( the Seagates are selling for $44 right now), and create a RAID 0 off the motherboard to have a large 4TB capacity AND speed similar to the SSDs! WARNING !!! the odds of ONE drive failing are BIG....you would immediately lose ALL your data !! HOWEVER....if you back up that RAID 0 RELIGIOUSLY....you would have NO worries. One way to do THAT is to buy a 4TB ENTERPRISE level 7200rpm HDD....Seagate has one with a 128 MB cache that does over 200MB/sec. for just over $300.
Even with all this covered, there are MANY factors which can CHOKE a fast computer....the CODECS you are using have a HUGE bearing on performance. The VERSION of After Effects is important...NEWEST versions have the BEST performance with features like the "global performance cache", etc. The use of "third party plug-ins" can CRIPPLE a machine if these are NOT written to take advantage of "hyperthreading",OR, are "32 bit" and HUMBLE your machine because AE is NOW 64 bit!!
Make sure you have Windows fully updated and the newest drivers for your video card. Then , make sure your Windows POWER SETTINGS are set to MAXIMUM PERFORMANCE when"plugged in"....the DEFAULT Windows settings...even under "high performance" has setttings that will THROTTLE your GPU and CPU if you don't change them.....I found THAT out the hard way!
Finally, make sure , in AE settings, you have set the"memory allocation" correctly...there is a formula regarding how much to assign on a "per CPU core basis". There ALSO is a selection to make for "allowing multiple frames" to be processed...make sure THAT is set correctly.
Good luck !! I hope this helps!
Every time I have to wait for a RAM preview to complete, I can't forget Steve Forde's tantalizing question from early this year:
“What if we did NOTHING else in After Effects during 2014 other than make it faster? I mean MUCH faster. I mean much faster without a specific hardware requirement (new CPU, GPU, disk, machine, etc)? ”
Absolutely positively wishful thinking on my part, but what if Adobe has been redesigning AE's core to this end? My impression has been that disk cache, while a definite improvement, has been a kludge applied to an increasingly obsolete architecture. I'm simply thinking that the decisions I might make today regarding the "ideal" PC build might be very different if Adobe has indeed been working on major renovations of AE internals.
Regarding drive setup, it is really best to test various configurations with your projects and your media on your exact system. That being said, I would try the following as a starting point:
- boot drive (240GB SSD) - fine
- 2x 1TB SSD RAID 0 - all files except media and render outputs
- 2x 3TB RAID 0 - media and render outputs
If you are running CC 2014, then you are licensed to try Adobe using Windows. You might find that running AE on native Windows would be faster.
Thank you for that detailed response JFPhoton!
While I understand the variables of which plug ins are being used, which codecs are being used, etc. can drastically effect performance, trying to narrow it down to drives and hardware for now. Though I am rendering out in Mac OS, predominantly in Prores 4444, which I believe requires the use of a 32 bit quicktime component that might be slowing rendering down...
While I definitely agree that CPU/RAM matters a lot when it comes to AE performance, the main area where I am looking to see a speed increase is in real time interaction with AE.
I want large projects with many layers to navigate faster without hanging. I want to be able to move very large images about without seeing any pixelation or lag in my previews.
My understanding is that the GPU and drive speed are what effect these rather than the CPU/RAM which help more with RAM previews and rendering performance.
Am I pretty much correct in this assumption?
Otherwise, to answer your question:
My current setup uses:
120 GB 840 PRO SSD - Boot drive
120Gb 840 PRO SSD - AE caches
240GB SSD - Project files and media
3TB HDD - Extra media when the 240GB SSD can not hold all needed media for a project, and often the location of where my renders are being written
As I mainly work on FX shots and motion graphics I do not need a huge amount of space as an editor might. But you are suggesting that having a dedicated cache drive is the best...
SSD Startup drive
2 x 120GB SSDs in RAID 0 - Cache?
2x 1TB SSD in RAID 0 - Media/projects/renders
Possibly another HDD RAID 0 for renders /larger projects?
My main thought is that I already have individual dedicated fast SSDs being used for most things but am still seeing more lag than I would like when loading from caches (blue bar) and in working with large uncompressed videos...So...I need to speed up my cache or media drives via RAID 0-ing them. Sound like a good idea?
I really think there is a gap in the market here for someone to create a streamlined workstation PC/MAC/Hackintosh build for working specifically in AE, now that it has grown so much and most peoples' growing pains are with regard to performance. I'm getting to the point where I would happily pay a premium to have an AE workstation that runs like lightening without all the fuss!
As I said before....I believe your CPU, memory, and GPU are all FINE !! Any slowdown is probably related to DISK SPEED...OR....the use of the "32 bit Quirktime" codec. The newer versions of Premiere Pro FIXED the 32 bit problem somehow and I noticed a MASSIVE IMPROVEMENT in performance using Canon MOV files which used to CHOKE my machine with the 4 GB limit of memory use, etc. However, I am NOT sure if this "fix" was ALSO applied to AE. If it WAS, using older versions of AE may still be troublesome with 32 bit Quirktime codecs.
In your description of your "current setup" you say the 3 TB drive is " often the location of where my renders are written " Do you mean EXPORTS ??? "Renders" means PREVIEWS !!! All previews AND "the global performance cache" should be on that second 120GB SSD !!! That 3 TB drive MAY have a slow performance.....slowing EVERYTHING DOWN if you are using it to place "previews on",or, to feed video into your AE projects, ESPECIALLY if using multiple layers of uncompressed footage !! Do NOT place any previews there !!
Is the 240 GB SSD a QUALITY SATA III SSD like the first two ??? For Video work ONLY use the Samsung Pro,or, the Crucial M550 SSDs....others SSDs have different controllers that have horrible write speeds and have trouble with incompressible data,which most digital video IS !!
Your 3930K overclocked should be able to easily handle multi-layered compressed video formats natively......using UNCOMPRESSED, HUGE FILES will tax the speed of your drive system !!! You need to know the "Megabytes per second" required transfer speed of this material to know if your drives can handle it. Most users use uncompressed when their CPU power is WEAK....you do not have this problem. Multi-layered compressed files should move EASILY on your SSDs. Just try working on those SSDs ALONE and do not use the HDD for now. If you are using the newest AE CC 2014 version, any problem may then be narrowed down to the codec you are using. However, there is only SO MUCH LOAD you can put on a system before it "chugs" Use monitoring software to see the real time use of your CPU,GPU,memory and DISKS so you can SEE if any of those components are fully saturated at their limit.
Adding video tracks will increase the drive speed needed to feed the video without stuttering. Adding LAYERS in AE will increase your memory usage and CPU load to process,BUT, you have great memory and CPU power....you should have no worry there.
Thanks again JFPhoton.
To clarify, my AE media cache is currently set to a dedicated 120GB SSD (840Pro). It gets good read/write speeds, often above 500mb/s when tested, SATA 3, etc., as do all of my SSD drives. The other is an OWC Mecury Extreme or some such.
The slow HDD is only used to render out finals to (exports if coming from premiere land) not previews but actual renders, when you let AE render your whole time line, sustained write. I get speeds of 150 mb/s or so via that 3TB drive so figured that is not the problem but wil of course try to keep everything as as many fast drives as possible. I'm just wondering how many fast independent drives do I need ideally?
I am using AE CS6 but am licensed to use CC via my work so might try that at some point as you suggest. I did for a short while when it first came out but did not see any speed changes. However, do not think quicktime has been changed so will still be using Apple's 32 bit quicktime helper when writing to Prores formats, even in CC? I've not noticed a drastic speed difference when using different codecs in AE, but do notice a speed drop when working in larger frame sizes, 2k-4k, again, which suggests the problem is drive speed rather than codecs. Again, more worried about real time working performance than rendering performance (ie, rendering OUT of AE, the only time it is writing to a different codec)
Realize that Adobe have always changed the way AE consumes memory as evident from the ever changing AE memory preferences. Does CC really have much of anything different from CS6 in this aspect?
So you are still suggesting that it is RAM and CPU that will be slowing down my actual real time AE performance? (working with large image files, or many layers in a comp...)
I find that frustrating as I have 64GB of 1600 speed ram and a 6 core CPU 3930k and I am still not seeing great performance there. I do indeed see the memory get used via activity monitor to the point where I have to force a 'purge all' every now and again to speed up my ram previews, and the CPU only really maxes out when rendering using multi-cores, not in real time moving layers and navigating the timeline.
Only other idea I have is that maybe I need VERY fast drives, as in over 600 gb/s speed on several drives. Hence my attempt to make several RAID 0 arrays out of SSDs... That and an upgrade to a gtx 980 from my older 660ti might make a change...
But overall, thank you and the others for the tips, lots of things to look into once again as there are so many variables. Will try CC again rather than CS6, also Windows rather than Mac, and probably go through a bunch of different drive configs and benchmark if I get the time. Will drop a line in this thread should I discover that there was one particular element slowing things down...
AE is not a realtime application like Premiere and other editors. Compositers like AE are built different and therefore don't have the same realtime capability. No matter what you change, to get realtime playback is going to require ram preview with AE. However more ram and cores/threads often equals greater ram preview performance which means faster ram preview. That is what you want. Setting the Adobe ram allocation in multiprocessor settings to 3 to 4GB per thread and enabling multiprocessing for ram preview should significantly increase the ram preview performance. AE CC has a much larger ram profile than AE CS6 so you will use more ram in general. Also When ever AE uses all available ram for generating the ram preview it disk caches the preview and clears the ram buffer before continuing. That is why disk cache is so important. To give you an idea 30 seconds of 4K at full resolution preview takes about 96GB of ram in AE CC to generate the ram preview file. 64GB is going to disk cache atleast a couple of times while generating that preview.
...I JUST saw the following article, which MAY have an impact on your situation....using the Samsung 840 Pro SSDs. It looks like you may have to update the firmware for those two drives.
You mentioned your media is on an "OWC" SSD.....that could be "iffy".....performance varies GREATLY on SSDs regarding their use for video.....testing and recommendations from pros like Eric on the forum center on using the Crucial M550,or, the Samsung 850 Pro models, due to their superior controllers,( not "Sandforce" ), which provide the best performance for video work.
Today, Samsung announced it is introducing its "3D memory" to its cheaper EVO line of SSDs....boosting performance up to near that of its Pro line. However, AGAIN,I am not sure what controller would be in use with the new EVO and if it would eliminate the previous write speed problems for use with video.
Now that you have mentioned you are using CS6 AND also trying to edit 2K and 4K material.....we now have a horse of a different color. Eric reports a big improvement on the processing speed of 4K material with the newest CC version of PPro vs. all previous versions....I do NOT know if this would hold true for AE. J.E. Short on this forum recently wrote describing the huge difference of PARTICULAR 4K file formats...."RED" is uncompressed with HUGE file size, ( if I remember correctly ), requiring a very fast disk system, while other camera based 4K codecs are compressed and are easier to edit, as long as the CPU and memory are up to snuff.
Eric here has mentioned that attempting to edit multi-layered 4K is very difficult.....memory use may even exceed the 64 GB you have.....and ALSO that it would require a greater onboard video memory on the GPU for "frame buffering".....at least 4GB.
At this point, you may want to seek Eric Bowen's advice here on the forum as he can give you the best advice concerning what is required for your EXACT AE work flow, ( i. e. what codecs in use, how many tracks, how many layers, what effects,etc.). He is at ADK Computers...they specialize in building machines for video editing. He will know EXACTLY where your current problems may lie.
In general, CS6 is getting "long in the tooth"and STUCK where they left it. Advancements are continuing constantly with the new CC software to keep up with new camera codecs, new hardware,etc. There have been complaints on the forums regarding some incompatibility between the Mac OS and the Adobe software from time to time...and also, observations that performance under Windows is better. Hopefully, a few small fixes can improve your current situation !! Of course, you can always build a Haswell E supercomputer !!!
Thanks for dropping a line! JFPhoton has said many great things about your efforts to shed some light on the ideal hardware setups for making AE fly!
Of course I am not expecting AE to work at all like a video editing application and am well accustomed to waiting for ram previews and renders...
That said, my main 'beef' is with how my AE interface moves quite slowly, or slower than I would like it to considering the amount of money I have invested in hardware looking to make it move faster.
And the aspect I would like to see move faster is mainly the interface itself. Not RAM previews or renders, but literally scrolling through the layers list in my project can often drag. clicking and dragging images or video to reposition when rotoscoping or the like often leads to pixelation or lag, forcing me to inch my way along with the arrow keys, etc.
I am hoping this will be helped my my recently order graphics card update to a GTX 980? As for the CPU and RAM, I think I have those as maxed out as I can right now. Always thought more cores = faster performance, but I'm thinking the sort of tasks I describe above are more related to single core performance rather than multi core performance? As I am not doing any RAM previewing or rendering here. Simply navigating the AE interface of a large project or moving around large layers with real time mouse clicks and drags....?
As such, I am starting to think that it could be best to work mainly on an overclocked CPU that has great single core performance (more like a gaming machine) and send my renders to a separate Xeon filled beast? Lengthy RAM previews are really not that important to me as the duration of the material I make is usually below 2 minutes total, rarely previewing more than 10 seconds at a time...
Thank you for the update as to that fact that AE CC has once again improved its handling of RAM to speed things up. I will definitely be trying things out in CC rather than CS6 at the same time as I install my new hardware!
Slightly depressing as I just bought 2x 1TB Samsung EVO SSDs as they were at a great marked down price...Now I know perhaps why they were at that price! But hopefully the update will help!
As said, will definitely try working in CC more once I get the time to upgrade everything and see how things have changed performance wise.
And I would love to be able to buy a supercomputer that can just run everything perfectly! Unfortunately, there does not seem to be a clear/simple answer as to what to buy or where to buy it!
Previously I've worked with lots of cores and ram, but starting to think a machine that looks more like a gaming beast might be best in terms of navigating the interface, as detailed above...
I am based in the UK and would greatly appreciate any references towards folk who offer workstation building services?
I often find I have to wait a few seconds when starting a ram preview when using multiple cores with the messages 'background tasks initializing' when starting the first preview after loading AE, and then 'saving locked frames' or something similar appearing thereafter every time I press 0. Lagging for a few seconds on every press. If I take off the mult-core rendering it all starts instantly...Don't get me wrong, once it gets rendering it goes much faster...But if only looking to preview 3-5 seconds I am usually better off without the multicore render turned on.
Again, something that has probably been address in CC but I'm getting tempted towards aiming for better single core performance because of this. Same goes for video cards. Workstations cards being very expensive compared to their gaming counterparts and not appearing to offer many benefits....
Thanks again for the responses guys! Been dying to rant about this for a while! =]
The AE Interfacing is the OS drive and AE cache drives. That is where SSD's make a big difference. The OS drive as an SSD and the AE cache on the SSD raid 0 would be great although I would not expect any great performance in raid with the 840 Evo drives. Maybe they fixed that with the 850 Evo's since the controllers now are the same as the 850 Pro's for the top 2 drives. However keep in mind TLC NAND has a far lower endurance capability and rating than MLC does. So media drives wont have near the lifespan with TLC SSD's versus MLC SSD's. The Micron M600's are 340TB endurance drives where as the new 850 Evo's are only 150TB rated drives for the 512GB and 1TB or 75TB for the smaller EVO's. SSD's are quickly becoming you get what you pay for. That endurance means far less for OS than it does data drives.
Thanks for the confirmation of what hardware effects the interface performance Eric!
I have been booting from a 120GB 840 PRO and using another 840 PRO as a dedicated cache for the past year or so. Both in SATA 3 ports.
And still some petty annoying lag/draggin/etc. =[
So will be trying my most likely flawed idea of a RAID 0 of 2x 1TB 840 EVOs, a HDD RAID, doing the long put off upgrade to AE CC, and trying AE within Windows 8 instead of Mac OS.
Will post some results when ready!