13 Replies Latest reply on Oct 24, 2011 6:42 PM by MichaelBuieFilms

    I increased CPU utilization from 45-55% to 70-80% during renders

    MichaelBuieFilms Level 1

      Problem? Well, my Intel Core 2 Quad 2.4 GHz PC would max at near 100% during rendering, so I knew my CPU was the bottleneck.

       

      So, I built an I-7 950 Intel CPU with 12 GB of 1600 DDR3 Ram.

       

      My Mainboard:

      http://www.newegg.com/Product/Product.aspx?Item=N82E16813188039

       

      Card: NVIDIA GeForce GTX 470 (to ensure use of GPU acceleration)

       

      Drives:

      C: 1.0 TB 5400 (System/OS)

      D: 1.5 TB 5400 (Office/Data)

      E: 1.5 TB 5400 (Old projects I may have to revisit)

      F: 2.0 TB 7500 (Current projects)

      G: 2.0 TB 7500 (Cache/Scratch)

      H: Blue-Ray DVD Drive

      I:   DROBO FW800/USB with four 2.0 TB 5400 (5.4 TB usable) for Offline video project and critical files storage

       

      After setting up my I7 950, I was getting 45-50% CPU utilization during rendering or eek-ing out maybe 55% if I set Premiere Pro process to near real-time priority

      Now, I suspected my controller and drives may become the bottleneck with my new build... but really? 50% CPU utilization for rendering? What the heck?

       

      Doing some research and making the below-listed adjustments, I managed to get CPU utilization up to about 75% median (70%-80%) for BOTH export rendering AND sequence workspace rendering.

       

      1. Started by defragging C: (G: was empty)

       

      2. Changed PageFile to 1GB Fixed on C: and 24 GB Fixed on G: (Am upgrading Ram to 24 GB soon)

       

      3. Changed Adobe Cache Files (not database) from default C: drive location to G: (Edit > Preferences > Media )

       

      4. Changed Scratch Drive for Project from same drive/location to G: for Video and Audio

       

      My two fastest drives, F: is my "Current Video Projects" drive and G: I made into my Cache/Scratch drive

       

      ====

       

      Next step I plan in my quest for faster render AND better editing performance:

      - SAS Controller and 15,000 RPM SAS drives.

       

      But, that's for when I have another paycheck to spare. LOL!

       

      I hope this helps someone. Critique and comments welcome.

        • 1. Re: I increased CPU utilization from 45-55% to 70-80% during renders
          Harm Millaard Level 7

          You clearly demonstrated that your system was hindered by bottlenecks in the disk setup, a factor often overlooked but ohh so important. Thank you for posting this.

           

          With proper backup to your Drobo, you may consider raiding D and E into a raid0 and use that for media cache and previews. For pagefile you can remove the one on C:. 24 GB on your G drive is more than enough and there is no need to burden the OS with accessing two pagefiles on two drives and it might be even better to locate the pagefile on the newly created raid0 as the first file, so it is in the fastest part of the disk and not fragmented.

           

          Check Adobe Forums: Generic Guideline for Disk Setup

           

          Compliments on a well constructed disk setup! As you have seen and demonstrated here, a good disk setup can make a world of difference in performance, even on a less than top-notch CPU. Or should I say especially...

           

          I'm curious to see how well your system performs when you run the PPBM5 Benchmark

          • 2. Re: I increased CPU utilization from 45-55% to 70-80% during renders
            MichaelBuieFilms Level 1

            Thank you so much for the additional advice about possibly enhancing the performance evem more with RAID options. I've been thinknig how I can utilize RAID to do that.

             

            The reason I left some PageFile on C: is because I've read some pagefile must be there to capture a memory dump if an unexpected fault ever occurred. Pelase read this or google it and tell me what you think:

             

             

            Complete memory dump

            A complete memory dump records all the contents of system memory when your computer stops unexpectedly. A complete memory dump may contain data from processes that were running when the memory dump was collected.

             

            If you select the Complete memory dump option, you must have a paging file on the boot volume that is sufficient to hold all the physical RAM plus 1 megabyte (MB).

            • 3. Re: I increased CPU utilization from 45-55% to 70-80% during renders
              MichaelBuieFilms Level 1

              Also, I Iook forward to trying the Benchmark tool ... I wasn't even aware of it

              • 4. Re: I increased CPU utilization from 45-55% to 70-80% during renders
                Harm Millaard Level 7

                Have you ever tried 'reading' a memory dump? It is utterly incomprehensible and no use for mere mortals. For me it is far worse than traditional Chinese characters, read out aloud by a Mandarin speaking person from Senegal, if any can be found there.

                 

                Memory dumps will be recorded on your C drive anyway, even if you have no pagefile there. I clean them ASAP, because I can't make head or tails from the info, but that is likely caused by the fact that I'm pretty simple minded.

                 

                In pingin 'ma ma ma ma' can mean lots of things, depending on the tone, but you can get the gist of it:

                 

                Tone 1: Ma = Mother

                Tone 2: Ma = Rope

                Tone 3: Ma = Horse

                Tone 4: Ma = Scold

                • 5. Re: I increased CPU utilization from 45-55% to 70-80% during renders
                  MichaelBuieFilms Level 1

                  LOL! Harm, you are too funny! Your Generic Guide for Disk Setup is great reading, BTW ... though I've yet to finish the whole thread. Thanks again.

                   

                  A memory duimp file can be used by support to help you figure out what may have caused unexpected crashes. There are even ways for mere mortals to figure it out if they want to:

                  http://support.microsoft.com/kb/315263

                  • 6. Re: I increased CPU utilization from 45-55% to 70-80% during renders
                    MichaelBuieFilms Level 1

                    Error.jpg

                    After running the tests for PPBenchmark test, I get the above error when I double-click the script. All 11 files are in the directory. Any ideas

                    • 7. Re: I increased CPU utilization from 45-55% to 70-80% during renders
                      Harm Millaard Level 7

                      The only thing I can think of is:

                       

                      1. Either the download got corrupted somehow, so try to download it again, or

                      2. You did not follow the instructions to the letter. It is a pretty dumb script and the smallest error is severely punished.

                       

                      Sorry for that, but remember that Bill and I are only simple nitwits, who have very limited experience with stuff like scripting or website design. Bill is the one who is mainly occupied with the script and giving individual feedback, while my efforts are mainly focused on the website and developing new test timelines for future versions and possibly porting to MAC.

                      • 8. Re: I increased CPU utilization from 45-55% to 70-80% during renders
                        MichaelBuieFilms Level 1

                        Everything executed as outlined int he document up to the point of running the VBscript.

                        I looked in the script and the file names seemed to match the files produced in the folder. But, it was late and I could have eyeballed it wrong.

                         

                        It will be fun to see what the score is.

                         

                        Thanks for responding!

                        • 9. Re: I increased CPU utilization from 45-55% to 70-80% during renders
                          MichaelBuieFilms Level 1

                          I posted my experience over at CreativeCow thinking someone there might want to see it since I saw some folk over there complain about why their CPU utilization is low on high-end machines.

                          Once guy told me my experience didn't mean anything.

                           

                          That is funny ... I love rendering 35% faster! Wonderful non-experience! LOL!

                          • 10. Re: I increased CPU utilization from 45-55% to 70-80% during renders
                            MichaelBuieFilms Level 1

                            OK ... my first time RAIDing a box, but I took the plunge.

                            I bought another WD Cavier Black (my third one), removed the Caching folders from my 2nd Cavier Black, and RAIDed those two RAID0.

                             

                            4TB hard drive space and HD Tach gave me these performance comparisons between the single Cavier Black and the two I RAIDed:

                             

                            Size: 2.0 TB 

                            Random Access: 11.7

                            Avg. Read: 119.6

                            Hi Read MB/s: 160

                            Low Read MB/s: 72

                            Burst MB/s: 241.8

                             

                            Size: 4.0 TB RAID 0

                            Random Access: 9.5

                            Avg. Read: 192.5

                            Hi Read MB/s: 221

                            Low Read MB/s: 140

                            Burst MB/s: 390.6

                             

                            WOW!

                             

                            Then, I tested render times and the RAID0 storage was quick enough that I could retain the same performance with Media Cache and Preview files pointed to the same drive as the Project Files. The Static Swap File is there, too.

                             

                            I tried keeping them separate as in my first optimization that, as I reported, allowed full CPU Utilization and found no real performance gain in doing so. So I am keeping them all on the RAIDed drive and thinking how I may use my single 2.0 TB Cavier Black. I may use it as a backup for currently running projects; I don't know. Or, I may buy another 1 TB drive and RAID 0 it with C drive to increase performance there. OR ... I might make C more safe and RAID 1 it.

                             

                            When I double-checked the actual drives as I RAIDed, I notice got a couple drive types wrong in my first post: I had already upgraded my C: to a 1TB Cavier Black, and my E: drive is a 1.5 TB 7200 RPM Seagate Barracuda.

                            • 11. Re: I increased CPU utilization from 45-55% to 70-80% during renders
                              MichaelBuieFilms Level 1

                              Opening a project with many video files is NOTICEBLY quicker. One thing about having the drives the way they are currently set up, I can duplicate projects and directly compare performance.

                              • 12. Re: I increased CPU utilization from 45-55% to 70-80% during renders
                                MichaelBuieFilms Level 1

                                OK, Harm ... I was able to get the Benchmark test to run.

                                 

                                "104","85","87","7" (MPE)

                                "104","85","87","108" (No MPE)

                                 

                                What happened the first time I failed was I could not have the Scratch folder on another drive.

                                I had swapped my Scratch folders to the same drive since it seemed to run about the same with it all going to the RAID 0 disk.

                                 

                                I did figure out haow to make it successful on split drives, though. To get around that, one can run the test with scratch folder on another drive, then just copy the rendered files to expected directory structure on the project drive when they run the VBS script.

                                 

                                Thanks!

                                • 13. Re: I increased CPU utilization from 45-55% to 70-80% during renders
                                  MichaelBuieFilms Level 1

                                  Hot DAY-yum!

                                   

                                  Two Cavier Blacks RAID 0 for cache/scratch and two VelociRaptors RAID 0 for current projects.

                                  Rendered HD video (albeit 720p) to a file on opposite drive.

                                   

                                  A Picture is worth a thousand words!

                                  Full_CPU_Utilization.jpg

                                  DANG that was quick!

                                   

                                  Now I can fully utilizing my system's CPU. I'll just set my system to throttle back on hte processing speed if temps rise too high for a long rendring project ...

                                  (Or, just render to a slower drive ...LOL!)