33 Replies Latest reply on Apr 11, 2011 10:13 AM by Bill Lake

    How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64

    Bill Lake Level 1

      I'm not seeming to get full utilization from my machine when rendering in Premiere Pro CS5. When I render media to file, and it's taking a while, my CPU and memory seem under unitlized. Using the PC task manager I can see that there's some optimization to use the machine better, but I'm not sure how to tweak it.

       

      My Machine:

      Quadcore 6850 3.0 (capable of being overclocked to about 3.3- but I haven't tried since loading Win 7)

      8GB RAM

      My Source Media is first transferred to two SATA 7200rpm disk in RAID0

      My Scratch is on a separate, but similar RAID0.

      My OS is Win 7 x64 and my apps are on the C drive which is a pair of 10Krpm in RAID0. I render final media to this drive, as well.

      I don't think I have any read/write to disk bottlenecks.

       

      With most closable apps closed in the background, when I render a big file, the CPUS are running 80-90% and memor is used 4.57GB of 8: (4.6GB in use, 2.6GB Standby, 1GB Free.) Very steady RAM use.

      Premiere Pro and Media Encoder set to use 6.5 of RAM. 1.5 reserved for other apps.

       

      Why isn't my machine using more of the RAM?

       

      QTx32 server is running in the background.

        • 1. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
          Crist OC/PC Level 2

          1- You have to config your CS5 for full use of the processor.

          2- you have to set CS5 to use maximun RAM I guess in your case will be 6.5GB as the PP let some free memory for the OS and others programs.

          2- Exporting will demand more RAM that a Render.

          3- Your virtual memory have to be set @ min:12288 max:16384

          4- Your HHD config is horrible. Use 1hd 10.000 rpm for the OS., 3HD R0 for Proyect and 1 for backup.

           

          B.R.

           

          Cristobal

          • 2. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
            Harm Millaard Level 7

            Have you ever watched a Formula 1 race and wondered how they can achieve those miraculous lap times with mind staggering  speeds?

             

            The reason is actually pretty simple. All the properties, features and capabilities of a Formula 1 car are finely balanced and create a synergy that enables them to achieve those impressive speeds. They have the engine power to accelerate very quickly, they have the gearbox to shift gears very quickly and maintain maximum torque, they have the brakes that enable them to brake very quickly and at the last moment before cornering and they have the grip from their suspension, tires and down force to keep the car on the ideal track.

             

            As long as everything is finely tuned and in balance with all the other components you have the capability to win races. Of course external circumstances, like rain and temperatures have enormous impact on race results, using slicks or intermediates or wet tires have a huge impact, as do the adjustments to front wing that are required to keep everything properly balanced. And of course, if your grip is gone or one of your gears is malfunctioning or your tires are gone or your brakes are overheating, you can no longer win.

             

            A computer is not really different. To get optimal performance, all the components need to be properly balanced. The computer is only as fast as the slowest component. And the computer is also faced with external circumstances, like rain and temperatures of the track, that you have to live with, but that have a major impact on performance. That is the codec of the material you want to edit. Then also take into consideration that the different circuits in Formula 1 require different setups for the cars, Barcelona, Spa, Monte Carlo, Monza, all require different tuning and setup of the car. Similar to the editing workflows of an editor. Long form or short form, multicam or not, few or many tracks, DL to AE, all these aspects have an impact on performance.

             

            In your case your system is not very well balanced. It is somewhat similar to the Formula 1 car that started on a dry track with slicks and it is now starting to rain and his opponents are all on intermediates. He now needs to brake earlier, accellerate less agressively, because his grip is gone. Why would your PC use more memory when it is waiting for the hard disk to write the results or the CPU is busy with housekeeping tasks?

            • 3. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
              Bill Lake Level 1

              Crist/Harm - I appreciate the responses, but I'm not understanding what to do differently. Believe me, I spent hours and hours researching how to set up my editing machine and I wish the information were more prescriptive. I do understand the goal is to remove bottlenecks and I do understand the hardware pretty well. I don't understand the editing software processing tasks well enough to know where the throughput is required if you think that's the problem.

               

              There's somethings that I should probably clarify about my situation, but I'd really like some specific, detailed advice.

               

               

              To clarify my situation:

              I have 3 fast drives: C, D, and E. Each is a pair in Raid 0 sata each with multiple channels. I'm not open to changing the configuration but I am open to where I scratch, etc.

              C 300GB: runs the OS (Win7 x64) and Apps. I'm not open to changing this.

              E: I move all my source footage to this drive from a portable hard drive that I get from the videographer.

              D: I use it for scratch.

               

              After working on a project, I backup to portable disks and I dont' want to talk about backup.

               

               

              Also: Crist said:


              1- You have to config your CS5 for full use of the processor.

                        - How? Exactly.

              2- you have to set CS5 to use maximun RAM I guess in your case will be 6.5GB as the PP let some free memory for the OS and others programs.

                        - What should I do differently? I have set 1.5G for other, the rest for Premiere Pro and Media Encoder (and in this case I was only running media encoder after I sent the job to the queue.)

              2- Exporting will demand more RAM that a Render.

                      - I was actually referring to exporting, so I may have used the wrong term when describing my problem. I don't have any problems in the NLE. My only problem is trying to optimize export.

              3- Your virtual memory have to be set @ min:12288 max:16384

                     - This is a new suggestion - where do I learn to do this or where do you mean to do this? In the OS or in one of the apps.

              4- Your HHD config is horrible. Use 1hd 10.000 rpm for the OS., 3HD R0 for Proyect and 1 for backup.

                    -  I have 3 fast drives that I am unwilling to reconfigure due to hw restraints. That said, I do think there's probably a better way to put the various tasks on the various drives. I do understand disk throughput, so I don't need a lesson there, but I have read everything I can possibly find on Premiere Pro disk useage and I'm still a little unsure. For example, if my source media is on one disk, my apps/os on another - where do I put Exports and where do I put Project to optimize read/write. I thought the project was low I/O so I put it with the OS/Apps and I assumed (and I'm unsure of this) that export should go on the C drive because I assumed the D (scratch) would require a lot of I/O. I suspect this is not an ideal use of my drives.

               

               

              • 4. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                Crist OC/PC Level 2

                Dear bill.

                 

                The reaplay to all your dout is her: http://ppbm5.com/

                 

                Use this as the best tool for test your pc In a video editing atmosphere.

                 

                 

                Base your reasult on this Benchmark and you will see you will get what you want.

                 

                B.R.

                 

                Cristobal.

                • 5. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                  Bill Lake Level 1

                  Can anyone provide more specific advice on how to better use the drives that I have for the various I/O chores that impact exporting media from Premiere Pro? I understand I need to optimize the storage, but I'm exhausted from the research and no better off. I don't want general advice, thank you, I understand I/O, but what I don't understand is what to tell Premiere Pro to use as the locations for the various chores.  I decided to rephrase the question so that I might focus the responses:


                  My primary problem:

                  When exporting media from Premiere Pro CS5 I'm only utilizing about 4 of the 8 GB in my system, even though I thought I had set up the system, and Premiere Pro to take advantage of all the memory. No dropped frames. I've been told that I'm not using my drives right to optimize throughput (even though the CPU is running at about 90%) so here are a couple questions:

                   

                   

                  1) Regarding Drives: I need advice on where to store each of the types of files for exporting chores.  I have 3 drives. I can't change them due to hardware constraints. Each is a matched pair of Raid 0 sata, 7200 (My OS drive is a pair of 10000).  My source media is on one drive, and apps/os are on another.  Where should I put the scratch? Where should I export to? Where should I store project data?

                   

                  2) Regarding Memory: I added 4GB to get up to 8GB and I'm not seeing much of it being used. I am running a clean install of  windows 7 x64 with few system house-keeping tasks in the background. No security software, no user apps running except Media Encoder.  When I run only media encoder, my quadcore 3.oGhz is about maxed out and my memory is using about 4 of 8GB. I've reserved 1.5 for other apps in the Premiere Pro setting because sometimes I run Photoshop. The rest of the memory is idle/reserved. The processor doesn't appear to be waiting on anything- since it's running near 100%. If I'm not really running anything in the background, then why wouldn't the CPU try to use more memory so that it processes bigger blocks of information. I am feeling like I got no benefit from adding the RAM to get it to 8GB.

                   

                  My system: PC quadcore 6850  3.0Ghz, 8GB Ram, Win 7 x64 clean install, no helper/background apps running, no security software, Premiere Pro CS5. No dropped frames.

                  Can anyone make specific suggestions please?
                  Thank you.

                  • 6. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                    Bill Lake Level 1

                    Crist

                    Regarding Paging:

                    When I change virtual paging, I'm given the option to set it for each drive. Should I use your advice to set that size range for each drive?

                    Bill

                    • 7. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                      Bill Lake Level 1

                      Crist,

                      I made the paging file changes you recommended and as a result memory utilization was worsened slightly  from steady at 4.57G to steady at 4.51G used.

                      I have Premiere Pro CS5  Preference>Memory: Ram reserved 1.5 (lowest). Ram shared with Pr and Adobe Media Encoder only. Optimize for MEMORY.

                       

                      Where should media cache files and database be stored?

                      • 8. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                        Harm Millaard Level 7

                        Set it to Optimize for Performance.

                        • 9. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                          Bill Lake Level 1

                          I optimized for performance, which put the CPU to a steady 100%.- Thanks Harm.

                           

                          Also, I monitored my disk I/O more carefully and the ONLY drive that has ANY processes waiting on it are my media source drive. Disk queue length fluctuates from 0 to around .05 and active time are peaking at 5% (Windows resource monitor). The other drives never go above 0.

                           

                          These seem like really low stats to me, but I'm not expert here.  I'm still only using half the memory. I have a feeling there's some windows setting keeping memory in reserve.

                          • 10. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                            Harm Millaard Level 7

                            Notorious resource hogs are indexing, anti virus and disk optimizers, plus other background processes. Check with Process Explorer and if you have more than 40 - 50 processes running, it is time to clean up. See this article, which also applies to Win7: Adobe Forums: Guide for installing and tuning a Vista...

                            • 11. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                              Bill Gehrke Adobe Community Professional & MVP

                              Bill

                               

                              It turns our that the real sweet spot for Adobe Premiere CS5 is slightly over 12 GB, it has been proven many times that the CPU efficiency improves when you exceed 12 GB of RAM.  This is even on the best tuned systems.  The most CPU intensive test of PPBM5 is the MPEG2-DVD encoding.  Just going from 12 GB to 14 GB has produced significant reduction in the processing time of this test.

                               

                              P.S. my third place score (number 1 single processor score) on the PPBM5 page is achieved without ever dedicating amounts of RAM, I just let it run and have Windows do it's thing.

                              • 12. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                Bill Lake Level 1

                                Bill Gehrke: Any ideas what would be preventing my system from actually using more than 4.6GB when disk-related throughput appears to be good (not perfect, but good) and 4 Cores running 100%? Does Win 7x64 reserve some portion for a reason?

                                 

                                 

                                 

                                Bill Lake

                                Executive Vice President

                                ClicFlic

                                703-349-3070

                                bill.lake@clicflic.com

                                • 13. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                  Harm Millaard Level 7

                                  Bill,

                                   

                                  One of the reasons could be that the codec is more CPU or disk intensive than memory intensive. Or backgroung processes/services are messing up. Hard to tell without the details.

                                  • 14. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                    Bill Lake Level 1

                                    Now THAT makes sense.  There are no background processes running that Adobe and Win7 can't live without that I can tell.

                                     

                                    My source video is uncompressed animation. My export is going to H.264 in an MP4, VBR5-6, Audio is going to 92kbps 44 (it's just narration.) max render quality. No script rendering. I didn't suspect this, but you may be on to something...

                                    • 15. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                      Bill Gehrke Adobe Community Professional & MVP

                                      Bill,

                                       

                                      If you would just run the PPBM5 benchmark and submit it, I and Harm both will get copies.  Since you are a local call for me I will be able to give you immediate feedback looking specifically at the H.264 encoding result to see if we can suggest any help.  I will be out tomorrow from 9:30 am untill 2.pm

                                      • 16. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                        Crist OC/PC Level 2

                                        I guess you got all info you need right?

                                         

                                        Good Luck Bill.

                                        • 17. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                          Harm Millaard Level 7
                                          There are no background processes running that Adobe and Win7 can't live without that I can tell.

                                           

                                          You will be surprised to see how many processes are running in the background if you check it. Go to Sysinternal Utilities and use Process Explorer to show you all the processes that are running. If the number is more than 40 - 50, it is time to kill a number of them. They all eat resources, use memory and CPU cycles and generally degrade performance.

                                           

                                          My source video is uncompressed animation. My export is going to H.264 in an MP4, VBR5-6, Audio is going to 92kbps 44 (it's just narration.) max render quality.

                                           

                                          This implies that large amounts of source material need to be read from disk (uncompressed is large) and then, using complex H.264 algorithms, needs to be heavily compressed. This is very disk and CPU intensive, but the resulting file is relatively small in comparison to the source material so that does not occupy much memory. The bottleneck is twofold. Getting the large source material off the disk, but that is a rather sequential step, provided you have relatively long animation clips, unless your disk is fragmented and then the computationally difficult compression to H.264 with MRQ on. That is quite a burden for the CPU.

                                           

                                          If the animations are QT animations, you have another complicating factor, the use of QT-32, which effectively kills the 64 bit nature of CS5 and degrades effective memory use.

                                          • 18. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                            Bill Lake Level 1

                                            I'm still working it, but getting closer, thanks. I'm going to run the benchmark.

                                            • 19. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                              Bill Lake Level 1

                                              Harm, Excellent insight. This is helpful, thank you.

                                               

                                              I'll further check/reduce the background processes.

                                               

                                              No disk frag issues these files were the first added to a fresh disk. (I just rebuilt this workstation and started clean). But I'm pretty sure I'll get a bump in performance if I add another disk to that source-media drive (Raid 0) because I can see some wait-states on that drive.

                                               

                                              Yes - they are qt animations, (I think) and QT-32 service is running during export. I'm aware of the 32 bit vs 64 bit benefit loss (read your writings on that). The videographer shoots digibeta and exports using Avid's Media Encoder. I can't import digibeta and have no slots left to add a blackmagic card. I couldn't get any of his Meridien codecs to work on Win7x64 so after testing a dozen export formats, found animation was the only one that ported decent quality that Premiere Pro could accept. Since he runs it on a Mac, I assume it uses QT32, but perhaps I should quit the process just to be sure it's required.

                                               

                                              Also, I'll run the benchmark, hopefully later today.

                                               

                                              This is helpful, thank you.

                                               

                                              -Bill

                                              • 20. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                                Bill Lake Level 1

                                                Bill

                                                That's a generous offer. Thank you.

                                                I'll run it.

                                                Bill

                                                • 21. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                                  Bill Lake Level 1

                                                  I ran the ProcessExplorer and after suspending or killing 4-5 unneeded processes/services, I had 46 remaining.

                                                  Indexing is, of course, turned off.

                                                  • 22. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                                    Harm Millaard Level 7

                                                    Bill,

                                                     

                                                    That looks good. I haven't seen your test results yet.

                                                    • 23. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                                      Bill Lake Level 1

                                                      I've not done it just yet.

                                                       

                                                      Bill Lake

                                                      Executive Vice President

                                                      ClicFlic

                                                      703-349-3070

                                                      bill.lake@clicflic.com

                                                      • 24. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                                        Bill Lake Level 1

                                                        When I try to load the PPBM benchmark Pr project, I get a message: "This project contained a sequence that could not be opened. No sequence preview preset file or codec could be associated with this sequence type."

                                                         

                                                         

                                                        I am running recently installed (within last 2 weeks) QT Player (Pro) V 7.6.9,which is up to date.

                                                         

                                                        I tried to import the media into a separate project and had no problems loading these files:

                                                         

                                                        clip 01 /18.mpeg

                                                        ing85316_01.mp4

                                                        phil avchd 1. mts

                                                        phil avchd.2.mts

                                                        rothenburg.avi

                                                         

                                                         

                                                         

                                                         

                                                        Bill Lake

                                                        Executive Vice President

                                                        ClicFlic

                                                        703-349-3070

                                                        bill.lake@clicflic.com

                                                        • 25. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                                          Bill Gehrke Adobe Community Professional & MVP

                                                          Bill,

                                                           

                                                          The only thing I can think of is that somehow the project file got corrupted.  Try a fresh download and see if that sovles the problem.

                                                          • 26. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                                            Bill Lake Level 1

                                                            same error.

                                                             

                                                            Let me confirm the unpacking instructions:

                                                             

                                                            I unpacked to: C/Bill_lake/ppbm5

                                                            and ppbm5 contains a subfolder after unpacking

                                                             

                                                            Is this correct?

                                                            I'm running premiere pro cs5 win7 x64

                                                             

                                                            -Bill Lake

                                                            • 27. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                                              Bill Gehrke Adobe Community Professional & MVP

                                                              Bill, you should be doing it on your project file disk drive not on the boot drive.  All your PPBM5 files including Previews and Output files should all be on that drive (your 7200 rpm RAID 0 array).

                                                              • 28. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                                                JEShort01 Level 4

                                                                Bill,

                                                                 

                                                                I've run PPBM5 off my boot OS drive before.

                                                                 

                                                                Try as c:\PPBM5 and without running it from within a sub-directory. I seem to remember that you must not rename it from PPBM5, but don't quote me on that.

                                                                 

                                                                Jim

                                                                • 29. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                                                  Bill Gehrke Adobe Community Professional & MVP

                                                                  Jim/Bill, our directions try to simulate the Adobe minimum requirements (OS/Applications drive and a minimum of a second project drive).  This error message has me confused I am wondering if it is that late version of QuickTime which was recently loaded.

                                                                  • 30. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                                                    Bill Lake Level 1

                                                                    The odd thing is that I can load and preview the media into a project, I just can't load the sequence file.

                                                                    I didn't think that used the QT server.

                                                                     

                                                                    -Bill

                                                                    • 31. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                                                      Bill Gehrke Adobe Community Professional & MVP

                                                                      You will find that Adobe loads the crummy 32-bit QuickTime file if you need it or not.

                                                                      • 32. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                                                        Bill Gehrke Adobe Community Professional & MVP

                                                                        Bill, I just downloaded the most recent Quicktime and PPBM5 runs fine, but did you let Quicktime change associations when it installed?  If so you may have to go back and make some file associations back to Premiere.

                                                                        • 33. Re: How to max memory/CPU utilization in Pr CS5 quadcore 8GB x64
                                                                          Bill Lake Level 1

                                                                          I thought file associations were only relevant if you try to launch a file, not if you open it from another application.