16 Replies Latest reply on May 26, 2009 6:44 PM by the_wine_snob

    Not utilizing full CPU?

    JAREDKUPER@HOTMAIL.COM

      I've created a project with ~20 mp4 video clips, simple transitions between them, totalling about 1hr 20min. I then export to DVD and the encoding process is taking about 20 hours or more!

       

      It seems from reading other posts that this should be taking 4-9 hours or so with a setup similar to mine.

       

      So I checked the processor utilization during encoding, and it is only using approx 50%. Everyone else commonly reports very high CPU usage. Is there a setting that could be limiting the CPU usage? Why else could this be so low?

       

      My system is 3Ghz dual core processor, 4GB RAM, Vista Home Premium, one SATA hard drive with the O/S, Premiere Elements, and Scratch disk, one SATA disk drive with the source files on it, I have exited every program I can during encoding.

        • 1. Re: Not utilizing full CPU?
          the_wine_snob Level 9

          Some processes are heavily CPU intensive, some can utilize GPU (CS4 and the nVidia CX card comes to mind) and some are I/O intensive.

           

          I think that you will get *some* performance, if you move the Project and Scratch Disks to your second SATA HDD. Right now, your OS, your Virtual Memory, your progam and your Export + Scratch Disks are on the C:\. That is a lot of work for one drive. You might get some ideas from this LINK.

           

          Good luck,

           

          Hunt

          1 person found this helpful
          • 2. Re: Not utilizing full CPU?
            JAREDKUPER@HOTMAIL.COM Level 1

            Clarification:

            Disk 1 - SATA - contains O/S, Premiere 3.0, and scratch disk.

            Disk 2 - SATA - two drives in RAID0 - contains the project files and the source files.

             

            When I check the Task Manager, I am seeing CPU at ~55%, Memory at ~50%.

            Physical Mem Total = 3318 MB

            Phy Mem Cached = 1714 MB

            Phys Mem Free = 1 MB

            Memory graph = 1.94 MB

            Physical Memory listed at the bottom = 60%

             

            Since I have 4GB installed, and I haven't really seen more than 1.94GB in use with Task Manager, could the computer not realize that it can use more RAM?

             

            I expect to see a bottle neck SOMEWHERE, but so far not finding what I expected. When I check Resource Monitor, every few seconds my Disk graph jumps up over 1MB/sec, but not sure if that's at 100%. Also, it is the two disks in RAID0 where the major disk reading is occuring, so they should be fairly fast. The CPU and memory do not really have any noticable jumps up. How can I find my speed bottleneck if it's not the processor or RAM? Or IS it the processor or RAM and I just am to computer illiterate to tell?

            • 3. Re: Not utilizing full CPU?
              the_wine_snob Level 9

              What are the settings for your Windows Virtual Memory, the Page File? Where is it located and is it dynamically managed, or is it static?

               

              What are the Advanced Settings in Control Panel>System>Advanced>Settings (first two tabs)?

               

              What is shown for Physical Memory in Task Manager?

               

              Are you using the 3GB switch in your boot.ini?

               

              How is your RAID managed, i.e. onboard controller chip, managed by software, or separate RAID card?

               

              In Task Manager, please list the Processes running.

               

              Any/all of these can play a role in the overall performance.

               

              Good luck,

               

              Hunt

              • 4. Re: Not utilizing full CPU?
                JAREDKUPER@HOTMAIL.COM Level 1

                What are the settings for your Windows Virtual Memory, the Page File? Where is it located and is it dynamically managed, or is it static?

                It is static, set to 5000MB, on a small partition of the RAID0 drive

                 

                What are the Advanced Settings in Control Panel>System>Advanced>Settings (first two tabs)?s

                Well, the first two tabs are "Computer Name" and "Hardware" which I don't think you mean. Perhaps you are looking for the Advanced tab, Performance, Visual Effects tab? This one is custom with most options disabled. The Advanced tab has Processor scheduling set to Program priority.

                 

                What is shown for Physical Memory in Task Manager?

                Physical Memory sections shows:

                Total - 3318

                Cached - 1626

                Free - 9

                The Physical Memory Usage History is a straight line at about 55%.

                 

                Are you using the 3GB switch in your boot.ini?

                I am not using the 3GB switch. I hadn't heard of it before, but now that I'm reading about it, maybe I should be doing this?

                 

                How is your RAID managed, i.e. onboard controller chip, managed by software, or separate RAID card?

                RAID is managed on motherboard, ASUS M3A78-EM

                 

                In Task Manager, please list the Processes running.

                The list is:

                Adobe Premiere Elements (Memory 1,035,848 K, CPU 52)

                explorer (Memory 35,000 K, CPU 0)

                svchost (12 instances) (sum of Memory ~150,000K, CPU 0)

                audiodg (Memory 18,000K, CPU 0)

                - all Memory/CPU for below processes is very small -

                System

                SLsvc

                CCC

                csrss (2X)

                RHDVCpl

                taskeng (2X)

                Smc

                ccSvcHst

                services

                taskmgr

                MOM

                winlogon

                Atl2evxx (2X)

                gbwinvnc (2X)

                jusched

                lsm

                wmpnsfg

                ccApp

                dwm

                ehmsas

                ehtray

                MDM

                SmcGui

                smss

                wininit

                winlogon

                wmpnscfg

                • 5. Re: Not utilizing full CPU?
                  the_wine_snob Level 9

                  Things do look clean - not familiar with all of the Processes, but some of the common "culprits" do not appear. Nice job in keeping these to a minimum. Most people have 60-80 going. Maybe someone else can look down your list and find something that can be eliminated easily for NLE work. I just don't see any potential problems, but remember I do not recognize some of those.

                   

                  I like the Virtual Memory setting, with only one question: you mention a partition on the RAID 0. Why a partition? The reason that I ask is that Windows first sees your 2 HDD's as one (the RAID), then it sees your partitions as several. If accessing what it now thinks are 2, or more, physical HDD's, it's telling the controller to get the heads in a couple of places at the same time. You probably have a good reason for doing it this way, but it does throw up a yellow flag to me. Having the RAID managed by the MoBo chip is not the fastest/best, but is usually more than adequate. It does save on a US$400 RAID card with multiple channels/chips.

                   

                  You have provided me, and others, with food for thought. I'll keep looking over your data, but just see nothing, except for the Page File on a partition, that might affect performance.

                   

                  One test that I'd run would be a slight re-config on your Scratch Disks. I'd do a Save_As, or a Save_As_a_Copy for your Project. Leave your original totally alone. Do a test with the Scratch Disks set to "Same as Project." This will put them on the RAID 0. Then, perform the same operations and monitor the time. Unfortunately, you're investing hours into this, but you might get enough of a performance increase to justify it for future Projects. One way around the time, would be to do a "test" Project with shorter versions of similar Assets. Do all processing on that Project, as configured, and time the results. Do the Save_As, make the changes to the Scratch Disks, and process again. Any difference?

                   

                  Good luck, and thanks for gathering the data,

                   

                  Hunt

                  • 6. Re: Not utilizing full CPU?
                    JAREDKUPER@HOTMAIL.COM Level 1

                    Thanks for trying.

                     

                    I originally thoguht I had a good reason for making the small Temp partition on my RAID drive. I was going to use it as scratch disk or page file, but then I realized that it wasn't as great as I thought it was going to be and never got around to changing it. Now I have this little 5GB partition for no reason. I will move the page file to the main RAID0 partition after this 20+ hour encoding extraveganza is done.

                     

                    I will try your suggestions, but will definitely have to do it with a smaller project. I can't run too many 20+ hour experiments, I have things to do!

                    • 7. Re: Not utilizing full CPU?
                      the_wine_snob Level 9

                      I completely understand, regarding the Render/Transcode/Burn time. I'd attempt to find the most difficult of the Assets (if there is one) and construct the "test" Project around that. Use some of the same "features," as your current Project, and test it both ways.

                       

                      As for the 3GB switch, it has worked beautifully for some, but not so well for others. Save a copy of your .ini file, just in case. You can swap this out in Safe Mode, should you not see a positive performance increase. I use it on my workstation, but have not gotten around to testing it on the laptop.

                       

                      Good luck with your 20+ hr. Export and with increasing the performance of your machine. Like I said earlier, it looks lean and mean. I do not have any benchmarks to help, as I have not done any HD work. You might want to look on this SITE. Bill Gehrke was going to do a test file with HD material, but do not know if he's added that. His site is about benchmarking PrPro with various hardware and software configs. By using common Assets and performing the same exact functions, he's developed a database of the best configs. Unfortunately, it's set up for PrPro, but you might be able to get some ideas, from the top-performers in the database.

                       

                      Let us know how it goes,

                       

                      Hunt

                      • 8. Re: Not utilizing full CPU?
                        Ed.Macke Level 3

                        I use Process Explorer for my Task Manager, so I'm a bit rusty on the Task Manager labels, but to me the "1MB Free" of physical space seems like an issue!


                        What I'm wondering here is if you're actually using up all of your physical memory and Windows is swapping. Swapping, of course, absolutely KILLS performance and would explain why it takes 20 hours instead of 4.


                        But if task manager is telling the truth and there's really only 1MB of free physical memory, that's definitely a problem. Your mention of 1.94GB free makes me wonder if, for some reason, it's really only using 2GB and swapping like crazy. Doesn't seem to make sense, since it reports 3318MB installed.


                        On the other hand, it reports 3318MB installed, you say only a max of 1940MB are used, and it reports only 1MB available, so.... dunno. But something to look into?

                        • 9. Re: Not utilizing full CPU?
                          Kodebuster Level 3

                          I have a Dual Core Vista-32 box with 3gig,

                           

                          Any Physical Memory above 3gig will NOT be utilized by the Operating System.

                           

                          The OS will grab about 1gig and the balance (2gig) is available for applications.

                           

                          So if you have 4gig on a V-32  (or any 32 bit) OS, 1gig will never be seen by the OS.

                           

                          When Burning-to-Folder on my V-32, CPU usually pings around 70 to 80 percent, and Phys Mem sits at 2.5 gig plus or minus.

                           

                          Personally I don't care for MPG formats of any flavor cause I've seen nothing but grief with any version of PE.

                           

                          My assets are always DV-AVI Type 2.

                           

                          I suspect if you can convert to this format, you should see a dramatic boost in times (barring any inherent HW issues that are lurking).

                           

                          20 hours on your setup is no where normal...

                          • 10. Re: Not utilizing full CPU?
                            JAREDKUPER@HOTMAIL.COM Level 1

                            Still stumped.

                             

                            I ran an experiment where I exported by shortened 6min project to AVI and it took a reasonable amount of time (20min) and had 70-80% CPU usage, still around 50% RAM usage, and much higher disk activity. Seems ok, still not seeing anything utilized at 100%, but not bad.

                             

                            I exported the same shortened project into DVD and it took FOREVER. I gave up after the first menu was still being encoded after 15 minutes.

                             

                            So now time for the changes...

                            I used the 3GB switch, got rid of the separate partition for the paging file, it is now on the RAID0 drive, and updated all drivers for the motherboard, chipset, RAID, and graphics.

                             

                            When I re-exported the shortened project to DVD, there is no change, it is still incredibly slow. It's like my DVD encoder algorithm is limited, but my AVI one isn't bad. BTW, source files are in MP4, does that make a difference? My camera will only record in MP4.

                            • 11. Re: Not utilizing full CPU?
                              Paul_LS Level 4

                              You will find quite a large variation in CPU usage dependant on the video format that is being rendered or transcoded.

                               

                              Also what size are your orginal MP4 files... probably not 720x480... so if you export/burn every frame will need to be resized plus the transcoding will take a long time. Did you wait for the 20 hour burn to complete? The time indicated is not very accurate, usually the estimated time to burn increases but actual burn time is usually shorter.

                              • 12. Re: Not utilizing full CPU?
                                the_wine_snob Level 9

                                And, to add to Paul_LS's astute observations, some fairly simple Effects can increase Render-time astronomically. Add Neat Video, Three-way Color Correction and a few more, and a 1 hr. Render can shoot up to times measured in "days." In PrPro, just adding a few % of Video Noise Reduction can increase the time by 8-16x. That is why commercial editing houses use "render farms," to complete the task. Unfortunately, neither PE, nor PrPro supports render farms.

                                 

                                Hunt

                                • 13. Re: Not utilizing full CPU?
                                  Ed.Macke Level 3
                                  Unfortunately, neither PE, nor PrPro supports render farms

                                   

                                  Good thing, too! Then I'd feel the need to actually build a render farm.

                                  • 14. Re: Not utilizing full CPU?
                                    the_wine_snob Level 9

                                    Hey, that could be a good thing. Down in Mississippi (where I'm from), there are still farmers getting a federal subsidy to NOT grow indigo - a plant once used to make dyes, that has not been farmed in the US since about the 1850's. One late state Senator was getting about US$1.5M/year to NOT grow it. Maybe they would pay us to NOT start a "render farm?"

                                     

                                    Just a thought,

                                     

                                    Hunt

                                    • 15. Re: Not utilizing full CPU?
                                      JAREDKUPER@HOTMAIL.COM Level 1

                                      Ok, after some experimenting, it seems that most of the problem is the original video format. When exporting to DVD a project that is made up of .mp4 clips, the encoding duration is horrificly long. But, if the clips are converted to .avi file first (which also takes horribly long) and then then the project made up of .avi files is exported to DVD, it takes MUCH less time.

                                       

                                      In my example, a 1hr video made up of 20 .mp4 clips took approximately 22 hours to export to DVD.

                                      Exporting the same 1hr video to avi took approximately 7 hours.

                                      Exporting the above .avi to DVD took approximately 1.5 hours.

                                       

                                      Even if you have to do the two steps, it's still about 1/2 the time. If you convert the clips to .avi right away though, then you can save a lot of time in editing if you have to re-burn things.

                                       

                                      Still not exactly sure why it's not using the full CPU, but definitely have an answer as to how to speed up my exporting.

                                       

                                      I tried most of the above mentioned "fixes" (experiments) and the made minimal performance changes, but nothing too spectacular.

                                      • 16. Re: Not utilizing full CPU?
                                        the_wine_snob Level 9

                                        In the case of conversion to something that the NLE works with best, it is not just the time involved, but also the ease of editing. That counts greatly, as well, plus it can eliminate other problems down the line. There is a bit more at stake, than just the total time of Exporting/Transcoding.

                                         

                                        If your tests indicate that you are at your max., that is probably the best that you can do. Why programs do not max out CPU's is a question for the coders. I have not seen much published on how PE does this, but with PrPro (earlier version w/ SD test materials) CPU's are heavily used, all cores, and HT, if available. Bill Gehrke has done a study on benchmarking performance with PrPro. The database of results might be of use.

                                         

                                        Hunt