26 Replies Latest reply on Mar 4, 2014 2:06 AM by Pierre Devereux

    Hardware Query...

    Pierre Devereux Level 2

      Hi all,

       

      Please allow me to start with an apology:

       

      I know this topic has been gone over again and again....and again, and I have read many posts around it, but I feel that every project has its own sepcial needs, and have seen time and time again, that certain effects require different power components. I have spent a lot of time over the last year or two figuring out what certain effects do, and trying to learn the ins and outs of certain After Effects aspects, but the one thing I have not looked into, to date, is what effects require which components to be powerful.

       

      So, saying that, I would like to explain our issue here at the office, give a run down of the effects we use and the hardware specs we have. I am in the position where we have the budget for one more PC, and it can be a pretty good one, (saying "Pretty Good" is of course relative) and I am terrified that I am going to buy a *SUPERCOMPUTER* - and end up with a PC that works beautifully fast, but processes the effects we use, the same was as our current machines.

       

      Here we go:

       

      Our footage is all 4K R3D RAW. I immediately render out 4K ProRes proxies using media Encoder on one of our Macs. I have set all the settings to as low as possible. These proxies are currently used on all other machines in the office for editing and adding effects. Once those projects are complete, they come back to me, and I replace the proxies with the RAW footage (or JPEG2000 intermediates) and take it from there. All our work is done in 4K, with the final 1080 output being done in Premier (to allow for any post movement).

       

      Our effects are pretty standard accross all projects, as we are working on TV episodes, they tend to use the same effects in different ways, the effects are mainly:

       

      1) Keylight

      2) Reverse Motion Tracking (as per VcP tutorial - http://www.videocopilot.net/tutorials/demon_face_warp/ )

      3) Auto Lip Synch

      4) VcP Optical Flares

      5) Stock Footage (anything from 4K to 2K to HD)

      6) Liquify tool (extensively)

      7) Holo-Matrix

      8) Trapcode Effects Suite (Particular, Shine, Lux etc)

      9) Layer re-timing (Either with Twixtor, or built in time stretching tools)

      10) Masking,Roto-scoping,Track Mattes layer transforms (the usual! )

       

      The Comps work pretty well on my PC, but as soon as I start stacking up the PreComps with the above effects, my PC starts showing great strain. I have the only PC in the office that has a Graphics card that supports CUDA. I have run a render, and done some tests using perfmon, and the funny thing is thast CPU, HDU and network are not that heavily utilized - (I know, it is a *point in time* test, so it might not be very accurate) and I am leaning towards thinking my biggest problem might be my Graphics card.

       

      Moving on to hardware, we run all projects over the network to our local machine (although, if copied to the desktop, there doesnt seem to be a visible difference). Footage files sit on a network drive, After Effects is installed locally on each machine. Here are hardware specs currently:

       

      My PC:

      Windows 7 - 64bit

      60GB internal SSD C Drive

      3TB External SATA drive connected via USB3

      32GB PC Memory

      Intel Core i7-3770CPU @ 3.4GHz

      NVIDIA GeForce GTX-570

      Disk Cache is enabled to the external drive

      3GB memory is reserved for other applications

      Render multiple frames simultaneously is disabled

       

      I do *most* work at quarter resolution, and thats not too bad, but often, when working on a shot, I need full res, and the other team members (being the artistic department) always require full res to judge that things feel right. Keeping in mind my PC works with the RAW files, and RAM Preview on projects containing a few pre-comps using some, if not all the effects mentioned above can obviously get quite hairy. I dont expect miracles, I understand Adobe software is designed and works a specific way, and this cannot change, I am just trying my best to figure out if I can upgrade to a better machine, will it improve performance significantly enough to make the purchase worth while.

       

      On all three other machines, the Prores Proxies are used, and the effects, for the most part, are:

       

      1) Reverse Motion Tracking

      2) Auto Lip Synch

      3) VcP Optical Flares

      4) Liquify tool (extensively)

      5) Holo Matrix

       

      The second PC works reasonably well. It has esactly the same specs as mine, except that it has one internal 500GB SATA Disk and the NVIDIA card is one iteration below mine (which does NOT support CUDA)

       

      Then come my biggest heartsore (and I am not going to go into too much detail why, because I dont want to spark a Holy War! ) - The two MAC's.

       

      One is used for mostly for adding Holo-matrix effects, the other solely for Lip Synch and Liquify tasks. These two machines struggle badly, Holo Matrix machine - usable, Lip Synch machine takes so long to RAM preview, it is pretty much impossible to judge a scene for quality.

       

      HoloMatrix MAC:

      OSX 10.8.5

      16GB Memory (1067 MHz DDR3)

      Intel i5 2.66GHz

      1TB internal MAC drive

      Disk Cache is enabled at 93GB on local disk

      3GB memory is reserved for other applications

      Render multiple frames simultaneously is disabled

       

       

      Lip Synch MAC:

      OSX 10.8.5

      16GB (1333MHz DDR 3)

      Intel i5 3.1GHz

      1TB internal MAC Drive

      Disk Cache is enabled at 200GB on local disk

      3GB memory is reserved for other applications

      Render multiple frames simultaneously is enabled at 4 cores, 2 reserved 3GB RAM allocated for bacjkground process.

       

      Both the MAC workstations get the full collected project on their local drives.

       

      And that, as they say, is that.

       

      I do understand that different people will have different opinions, but any advice on either a new power PC that should handle the specific effects like a dream, or even suggestions on component upgrading in the existing machines would be greatly appreciated - Have a great weekend all.

       

      Pierre Devereux

        • 1. Re: Hardware Query...
          Pierre Devereux Level 2

          Hi,

           

          In retrospect, I believe I have posted a rather unfair query. - But I am going to leave it up anyway, to get those who wish to comment's feelings - especially around the specific effects we use the most and their requirements.

           

          Todd,

           

          I have the links that you posted in various threads around improving performance, and I am reading through them and changing as I go - I promise!

           

          I am also, once again grateful for any extra comments here.

           

          Pierre

          • 2. Re: Hardware Query...
            Todd_Kopriva Level 8

            One point: You mentioned CUDA. That has _nothing_ tho do with After Effects outside of the ray-traced 3D renderer, which you did not mention using. Don't misunderstand that you need an Nvidia CUDA card; you only need that for After Effects for the ray-traced 3D renderer.

            • 3. Re: Hardware Query...
              Pierre Devereux Level 2

              Hi Todd,

               

              Thank you for taking the time to reply - I am sorry that I am asking the same thig that thousands of others have, I just really dont want to waste money on the budget we dont yet have!

               

              I am going to go through your previous posts step by step on the MAC today and see what happens. If there is no improvement, I hope that you will not mind me asking for further assistance here?

               

              Being an ex Server Engineer, my solution was always to throw memory and CPU at a problem to make it go away. Now I am a little out of my depth, but, if we were able to learn so much so far, I am sure we will get there.

               

              Thank you

               

              Pierre

              • 4. Re: Hardware Query...
                Todd_Kopriva Level 8

                > my solution was always to throw memory and CPU at a problem to make it go away

                 

                That's a good brief summary of what to do to make After Effects work faster. I'd add multiple fast disks to the list. SSDs for cache drives are an especially good idea.

                 

                For After Effects, don't worry about the GPU unless you're using the ray-traced 3D renderer. Whatever you're getting for your Premiere Pro work will be more than adequate for After Effects.

                • 5. Re: Hardware Query...
                  Pierre Devereux Level 2

                  Todd_Kopriva wrote:

                   

                   

                  For After Effects, don't worry about the GPU unless you're using the ray-traced 3D renderer. Whatever you're getting for your Premiere Pro work will be more than adequate for After Effects.

                  Hehehe,

                   

                  Currently my After Effects PC is the best one in the office. The 4K footage run fine on my PC, but not on the other. The only difference between the two was that mine has a GPU that is one level better than the other (so mine supports CUDA) and my C: Drive is a SSD drive.

                   

                  So it seems the SSD makes a really big difference, beacuse, other than those two, the PC's are identical. OK, the next step then, is to figure out what I want to do, test new SSD drives first, then up memory and CPU, or CPU and MEm first. May I safely go out on a limb to say that since we have 32GB of memory, and 1 Physical CPU with 4 Cores and 4 Threads, we should ook to invest money on SSD drives first?

                   

                  Pierre

                  • 6. Re: Hardware Query...
                    Todd_Kopriva Level 8

                    Yes, try some fast disks and run tests using those for caches and footage drives. You can get a lot of improvements for relatively little money in these areas.

                    • 7. Re: Hardware Query...
                      Pierre Devereux Level 2

                      Hi Todd,

                       

                      We are discussing getting a new PC, however, making some of the changes suggested on your posts to the MAC, and disabling some of the effects untill we actually need to evaluate them (and then just being patient!) seems to have brought the comps back to a "workable" state. It can definitely be better, but it is not as bad as we experienced last week.

                       

                      I am going to go ahead and post the specs of the proposed PC, just for the sake of thoroughness, please, if you do have a minute, would you mind commenting on it? Keeping in mind, that to date, the worst effect (Performance wise) seems to be Auto Lip Synch on a 4K footage item that has been reverse stabalized in a precomp. (We disable one of the specific effects that the plug in applies, and the machines performance increases substantially)

                       

                      Intel Xeon E5-2620

                      MB Dual socket LGA2011 support upto 512GB memory

                      32GB (DDR3 1600MHz ECC-Registered)

                      2 x 120GB SSD 1 x 3TB HDD (sustained data rate - 210mb/sec)

                      Nvidia GTX 770

                       

                       

                       

                      Will this, along with configuring the After Effects preferences, be a mchine worth spending money on?

                       

                      (I feel like I am asking you guys to solve world hunger here! so I do understand that I cant hold you responsible for suggestions - I just welcome the input)

                       

                      Pierre

                      • 8. Re: Hardware Query...
                        Pierre Devereux Level 2

                        Hi Todd, (and anyone else...)

                         

                        Are you able (or allowed) to make comments on a specific build of a PC? We want to source 9 new workstations for the new year, and I am very concerned that we buy them, and we sit with the same issues we have now - and that is that we cannot achieve real time playback - in AE or Premier, and effects (like auto lip synch) take so long to render each frame, it really is almost impossible to visualize what the scene looks like until it is rendered out - then any changes that need to be made take us back to the beginning.

                         

                        In case you are allowed to comment (and any tech-head readers want to comment), here are the specs I am looking at (and hoping it can be called a "Power PC")

                         

                        1) Intel s2600CP2 server Mother Board

                        2) 2x Intel Xeon E5-2620 V2 (6 core - 12 thread) CPU

                        3) 64GB RAM

                        4) Asus GTX770-DC2oC-2GD5 Graphics Card

                        5) 3X180GB SSD Drives (1 for OS, 1 for Cache and 1 for source)

                        6) 1x4TB drive for miscellaneous files and storage.

                         

                        Is this a worthwhile investment in the forums opinion?

                         

                        Pierre

                        • 9. Re: Hardware Query...
                          cc_merchant Level 4

                          Not really.

                           

                          1) Intel s2600CP2 server Mother Board: Pretty expensive with limited functionality. No audio, no fire-wire, no USB3, no eSATA.

                          2) 2x Intel Xeon E5-2620 V2 (6 core - 12 thread) CPU: Very low clock speed and two are pretty expensive.

                          3) 64GB RAM: As long as it is non-ECC memory, that's OK.

                          4) Asus GTX770-DC2oC-2GD5 Graphics Card: Looks like overkill, but OK. A 760 will suffice.

                          5) 3X180GB SSD Drives (1 for OS, 1 for Cache and 1 for source): The mobo only has 2 x SATA 6G, so the 3-rd SSD will be severely handicapped.

                          6) 1x4TB drive for miscellaneous files and storage.

                           

                          You would probably be better off with an overclocked i7-4930K, which is less expensive, gives more functionality on the mobo and often is significantly faster.

                           

                          Have a look at Tweakers Page - What kind of PC to use?

                          1 person found this helpful
                          • 10. Re: Hardware Query...
                            JEShort01 Level 4

                            Pierre,

                             

                            Your workflow, RED 4k RAW with numerous effects and more, is incredibly intense.

                             

                            I would suggest getting in touch with Eric at ADK for some help.

                             

                            As you have a need for 9 VERY expensive systems, I would strongly suggest you get the first one built and see how it works for your workflow and in your shop (once you have finalized the spec.).

                             

                            As CC_merchant points out a OC'd 4930k would be better than dual 2620 v2's. Additionally, it could be that faster cpus in a dual Xeon configuration would be better still. RED is wonderful and it is also a beast. I know that RED Rocket cards can help with some things, but I'm not sure if that applies once you have rendered that out to ProRes.

                             

                            Finally, I've only used RED (up to 4 layers) on the timeline in its natural highly-compressed state. Since the first part of your workflow is to uncompress it, you may have some serious drive bandwidth needs. And possibly one that a large RAID would serve much better than the SSD/drive mix that you mention here.

                             

                            Regards,

                             

                            Jim

                            1 person found this helpful
                            • 11. Re: Hardware Query...
                              ECBowen Most Valuable Participant

                              There are some definite considerations here that you want to look at.

                               

                              1. 1x Xeon E5 2697V2 on an X79 board that can handle the E5 V2 Xeons is a far better option than Dual 2620V2's at that clock speed. If you want 2x Xeon then dont bother with less than the 8 Cores and really the 10 or 12 cores. I would also suggest you look at a different Dual Xeon board.

                               

                              2. You need to make sure you configure enough ram for the threads with AE so the more threads available, the more ram you require for optimal configuration. 64GB is fine for 24 threads but if you go with bigger Xeons then you will need 128GB of ram and suggest Reg Dimm's there

                               

                              3. The Premiere work with 4K would really use a 780Ti or Titan card. That will have little effect in AE if that is the majority of the work on this system, then a 770GTX is fine.

                               

                              4. I would suggest a different config for the SSD's. A 180GB or higher will be fine for the OS. I suggest a 960GB Crucial drive for the media and cache though rather than smaller SSD's to split those. SSD's have such a low latency that 1 can handle both cache and media easily. 2x 960GB SSD's in raid 0 would be ideal with this workflow and what I would suggest over all.

                               

                              5. Red R3D threads out to as many Core/threads as the system can provide and requires allot of threads for 1/2 res ideal playback. You want 32 to 48 threads for ideal 1/2 res playback. Pro Res is a good performing Proxy overall if the Proxy workflow is required to maintain however Pro Res wont thread as well with those effects as Red would if you have the system to support the R3D. Pro Res really just allows that workflow on a lower end system than what would be normally required for 4K and R3D.

                               

                              Eric
                              ADK

                              1 person found this helpful
                              • 12. Re: Hardware Query...
                                Pierre Devereux Level 2

                                Guys,

                                 

                                Thank you so much for this. The information here is brilliant. I will take some time to process it and come up with a new build.

                                 

                                Reall, thanks again, I am so concerned that we spend so much, and the machines have very little added boost. At the moment, the AE work is most intensive, and only intermediate renders are edited in Premier - though the current PC struggles with even the intermediates!

                                 

                                Ill also look to building two seperate systems, one for the editors and one for the VFX team.

                                 

                                Thanks guys, ill update soon...

                                 

                                Pierre

                                • 13. Re: Hardware Query...
                                  Pierre Devereux Level 2

                                  Hi Eric,

                                   

                                  Sorry it has taken me so long to get around to relooking at our PC specs, and the question I asked previously, we have just finished our pitch and sent off to the relevant networks - now we hold thumbs.

                                   

                                  I sent a lot of the details here through to our IT source guy, and he took a look through. He believes that the "Warrior" system mentioned in the link sent by CC_Merchant would be a good one for us. the specs are as follows:

                                   

                                  PC Build.JPG

                                   

                                  This will then be the same spec used for both Premier and AE workflows. I am just busy thrashing out the best Hard Drive configuration at the moment. Does this seem a better option?

                                   

                                  Thank you again for your input.

                                   

                                  Pierre

                                  • 14. Re: Hardware Query...
                                    ECBowen Most Valuable Participant

                                    It will handle the job with Premiere easily though you may want the 770GTX or 780GTx. That config will handle AE. Ram Preview and Export may just take longer depending on workflow with the 4K. However you can get the work done so a good starting point. Just make sure you get a X79 board that can handle a E5 V2 Xeon incase you want to upgrade the CPU later to expand the workflow.


                                    Eric

                                    ADK

                                    • 15. Re: Hardware Query...
                                      Pierre Devereux Level 2

                                      Hi Eric,

                                       

                                      thanks for the feedback, the problem here, is that the bulk of the work is done in AE. DO you have any suggestions that would make the PC more suited to getting AE previews and renders faster?

                                       

                                      Pierre

                                      • 16. Re: Hardware Query...
                                        ECBowen Most Valuable Participant

                                        The 4K media is really what decides this along with the FX used. If you are talking Red here then a single 10 or 12 Core Xeon will far outperform the 6 core with AE if you have the multiframe rendering selected unless the FX limit the threading. That is what I would suggest with 64GB of ram.

                                         

                                        Eric

                                        ADK

                                        • 17. Re: Hardware Query...
                                          Pierre Devereux Level 2

                                          Hi Eric,

                                           

                                          In your first reply, when speaking about the Hard Drive configs, you suggest a RAID 0 config for cache and media, do you mean source file media, or export and ram preview media? As far as I can understand (AE specifically) at the moment, it is best to have O/S and Application on one drive, Cache on another, Source files on yet another and preview and export renders on another. Does this sound right?

                                           

                                          Pierre

                                          • 18. Re: Hardware Query...
                                            ECBowen Most Valuable Participant

                                            That depends on your drive setup and configuration. A raid 0 is fast enough for Media files and cache files. AE cache is always preferably on an SSD. If you only have the SSD for the OS and it's big enough then putting that AE cache there is ok since the latency of SSD's is so low. People separate the drives like you stated but often that is not necessary if you have the  right drives and config.

                                             

                                            Eric

                                            ADK

                                            • 19. Re: Hardware Query...
                                              Pierre Devereux Level 2

                                              Hi Eric,

                                               

                                              Just another quick one, Is it worth getting a larger SSD drive and partitioning it? Then storing source, cache and exports each on its own partition?

                                               

                                              Pierre

                                              • 20. Re: Hardware Query...
                                                cc_merchant Level 4

                                                No. Partitioning is always a bad idea. On mechanical disks because it increases mechanical 'wear and tear', on SSD's because it decreases effectiveness of the Trim function, thus reducing write speeds.

                                                 

                                                See Tweakers Page - Do's and Don'ts and specifically point 4 of the Don'ts.

                                                • 21. Re: Hardware Query...
                                                  Pierre Devereux Level 2

                                                  Thank you

                                                   

                                                  Thats what I wanted to hear. The hard part here, is the pricing of the SSD drives at the moment. you can get one 750GB SSD for the price of two 256GB SSD Drives. So a price and budget driven design, would be to get bigger drives, and use them wisely. The other alternative, is the 'safe' one, in so far as little configuration is then needed. In other words, sourcing three SSD drives for OS,Cache and source, and then a SATA for renders. however, in the long run, this is the more expensive option (if you work out cost per GB).

                                                   

                                                  So maybe, after reading and re-reading this thread, it should be one 750GB for OS and Cache, then one 256GB for source. and still the SATA for renders - we do our renders overnight, so a little bit of latency there is not an issue.

                                                   

                                                  Pierre

                                                  • 22. Re: Hardware Query...
                                                    Pierre Devereux Level 2

                                                    Hi,

                                                     

                                                    Even further reading, and looking at the history of our existing projects, One 256GB for OS and Cache (Disk and Media Cache) - 100GB for Cache, All-in-all and 156 Odd GB for the OS and Apps. Then just the one 3TB SATA. This is a bit cheaper - 2 SSD drives and a cheaper SATA drive.

                                                     

                                                    Pierre

                                                    • 23. Re: Hardware Query...
                                                      ECBowen Most Valuable Participant

                                                      Careful putting the Media Cache on the OS drive. Most dont remember or know how to delete those files and they go through SSD capacity quick. Also the firmware on the SSD drives are still not the best at cleaning the garbage up over time so when the OS SSD slows down you dont have the normal easy process to fix it.

                                                       

                                                      Eric

                                                      ADK

                                                      • 24. Re: Hardware Query...
                                                        Pierre Devereux Level 2

                                                        Ok,

                                                         

                                                        so, would it be safe to have the media cache and the media output - render location on one drive? - alternatively, clearing the cache folders once a week or so - depending on the project, or how far along on the particular project you are?

                                                         

                                                        Pierre

                                                        • 25. Re: Hardware Query...
                                                          ECBowen Most Valuable Participant

                                                          Yes you can put the media cache on the media drive if that is a raid 0 or SSD with large resolution media. I normally suggest cleaning the cache atleast once a month or between projects.

                                                           

                                                          Eric

                                                          ADK

                                                          • 26. Re: Hardware Query...
                                                            Pierre Devereux Level 2

                                                            Hi All,

                                                             

                                                            I am sorry to harp on this again, but with the exchange rate changes, and some component changes, the price of the machine I want to buy has almost doubled. If it is ok, can I lay out a final spec for approval by those "in the know"? :-) To be used for After effects and Premier.

                                                             

                                                            Lian-Li PC-A79 Case

                                                            Corsair AX1200i PSU (these two components alone are R10K :-(    )

                                                            Asus P9X79 motehrboard

                                                            Intel Core i7 4930K 6Core 12 threads 3.4GHz to 3.9GHz

                                                            Coolermaster V8 GTS Air CPU Cooler

                                                            8X8GB Corsair memory

                                                            nVidia GTX 760 GPU

                                                            1X256GB SSD for OS

                                                            1X1TB SSD for Cache and source

                                                            1X3TB Internal SATA for nightly Output renders

                                                            Windows 7

                                                            27'' Phillips monitor

                                                             

                                                            Well, there we have it. Price jumped from R39 205.91 to R61 312.18 - oh well, I guess we should've bought earlier!

                                                             

                                                            In your past experience, will a machine like this handle 4K footage and After Effects work.

                                                             

                                                            Thank you

                                                             

                                                            Pierre