10 Replies Latest reply: Dec 1, 2013 5:40 PM by JSS1138 RSS

    Help me identify problem for slow export on a powerful machine

    archie29 Community Member

      Hi,

       

      I am exporting an interview in Premiere Pro CC. It is 1:45:00 long and it has been 20 hours since, but the export is only 94% complete.

       

      Project & Footage:

      The footage is a combination of DSLR footage - 5D Mark III, GH3, 650 & 550D.
      The edit is all straight cuts with: {a} Color corrector on each clip to match all cameras {b} NEAT - reduce noise on around 10% clips {c} Sharpen (10-30) & {d} An adjustment layer overall with a 3-way color corrector for a simple color grade.

       

      Hard Drives:

      1. E: SSD-RAID for the source footage (Samsung 840 SSD on RAID-0)
      2. D: SSD for previews and cache (Samsung 840 Pro)
      3. C: SSD for Windows, Adobe CC & all programs (Samsung 840 Pro)
      4. G: Simple Seagate Hard disk for Export (used to be a 3 disk RAID but 2 drives crashed some time back)

       

      Other Machine Spec:
      Intel i7-3930K overclocked by 10%

      64GB RAM
      GTX-770 GPU

       

      This is my longest project to date in terms of total time, but last time I did a 1:00:00 project, took around 2 hours not 20.

       

      My concern is I need to understand where the bottleneck is?
      My CPU is hardly working a sweat - 40% at max

      My RAM is plenty - being used 40-70%

      GPU is not even being used in anything (see it at 0%)
      Disks (which I was sure was the bottleneck) barely cross 10MB/s

      Checked the "power options" in Windows also - everything is set to maximum performance.

       

      It was a big mistake not to send it to Media Encoder as I could have continued editing a different project. Biggest. Mistake. Ever! (no idea why Adobe does not make it a default behaviour?)

       

      I am fairly new to the industry and till last year was using a MacBook Pro until I started getting demanding projects.
      I want to know is this normal? If I had used NEAT video for 100% footage then probebly I could have understood this, but its used for only 1/10th of thr footage and I have used NEAT before (but on After Effects CS6) on my Macbook pro (nowehere near as fast as my current machine) and it only added a small overhead - nothing of this sort.


      I expected maybe 3 hours to export this 90-100 min seqeunce, not over 1 day.

      So what could be the reason? Why isn't one of the above (CPU, RAM, Disk, GPU) at 100%?
      I expected 1 of these to be the bottleneck but nothing is crossing 50% even when it should be on full blast.

       

      I dread what if there is small tweak required in the edit once its complete. This video is due on Monday, I would have to reencode for 24 hours.

       

      Please experts - let me know your thoughts. Could it be that Creative Cloud Windows is slower than CS6 Mac & its not a hardware problem?
      I am sure I am missing some hardware that is causing this bottlenck. Attaching resource Monitor screenshot (let me know if I should use some benchmark software for you guys to get more information).

      ResMon.jpg

       

       

        • 1. Re: Help me identify problem for slow export on a powerful machine
          archie29 Community Member

          UPDATE: My export crashed (and heart sank) after 96% (25 hours). Win7 restarted. I have no clue why (If someone has any tip how to seach logs/dump in Windows to find a reason how, please let me know - I was worried about CPU temperature but that was around 40 deg, I was monitoring it.)

           

          Anyways, so I used Media Encoder now (instead of direct export in PrPro) and made 2 changes:

          1. I unchecked "Render Maximum Depth" just as it is in the the default Youtube 1080p@24p preset (but kept the keyframing distance at 24 instead of 72 which is default).

          2. From VBR - 2 Pass, I made it CBR


          Its been 15 min I started the export and the video is 20% encoded already, with remaning time just 00:30 min.

           

          Now I am flabergasted.

           

          Did not using maximum depth & VBR- 2 pass alone make such a difference? From 30 hours to 1 hour? I am still using all the video effects and absolutely no change to the edit.

           

          I am seeing 2 differences the way my machine is working:

           

          1. Is Adobe somehow using some old renders it did for the last export? I am using disk utility and it is refering to original source files and some "Media Encoder preview files" which I think it was not using before (could be wrong)?

          So all that hard work it did for 25 hours is being utilized and if I do such a project again it will take me 30 hours again. Right?

           

          2. GPU is now being used (and to good effect). How is GPU being used now? I am not scaling or using any fancy effects. Just Neat Video, 3-way color corrector & Sharpen.

           

          This was before (export done in Premiere Pro directly):

          GPU.gif

          And this is now:

          GPU2.gif

          Very sad that no one (even from the Adobe Staff) has replied to my post. Did I say something inappropriate or TMI?
          C'mon guys I am on the Creative Cloud - show me some love

          • 2. Re: Help me identify problem for slow export on a powerful machine
            John T Smith CommunityMVP

            >Very sad that no one (even from the Adobe Staff) has replied to my post

             

            I can't help with your problem (different camera) but you do understand that most Adobe staff do not (as far as I know) work Saturday or Sunday?

             

            If some other user reads AND has an idea you will get help... but, maybe not as fast since this is on a weekend

            • 3. Re: Help me identify problem for slow export on a powerful machine
              NilsW Community Member

              The bottleneck could be the ssd for cache files. Try to delete all media cache files, let ppro generate new ones and then export. There must be a real problem. Of course 20 hours are not normal.

              This video should get exported in 2 or 3 hours.

              • 4. Re: Help me identify problem for slow export on a powerful machine
                JSS1138 CommunityMVP

                 

                Did not using maximum depth & VBR- 2 pass alone make such a difference?

                 

                It wouldn't surprise me.  Checking MRQ forces the render back into the CPU even if you have GPU acceleration on, and 2 pass doubles the export time.

                • 5. Re: Help me identify problem for slow export on a powerful machine
                  archie29 Community Member

                  Thanks John, wasn't aware of that.

                  • 6. Re: Help me identify problem for slow export on a powerful machine
                    archie29 Community Member

                    Thanks NilsW, I tried it but does not look like it made any difference

                    • 7. Re: Help me identify problem for slow export on a powerful machine
                      archie29 Community Member

                      Thanks Jim, I have read all threads regarding a balanced machine and spent all money on this.
                      So for a 2 hour edit with 4 multi-camera sequence and some Neat Video usage (frankly for 10-15 min of footage) can take 30 hours even with my machine (specs above) if done with MRQ and VBR?

                       

                      I had a busy week to get that project done but I want to do some "checks" how to identify the bottleneck. Can you suggest something? And what softwares should I use to check the benchmarks?

                       

                      I am now exporting some  7 minute videos of live music events with loads of Neat Video & Cinematic look adjustment applied that is taking 50 min for a 7 minute video with MRQ and 8Mbps CBR.

                      Is it a good time? I was hoping for something like 20-25 minutes.

                      • 8. Re: Help me identify problem for slow export on a powerful machine
                        JSS1138 CommunityMVP

                        Your best 'test' is the same project but with MRQ off and only using 1 pass.

                        • 9. Re: Help me identify problem for slow export on a powerful machine
                          archie29 Community Member

                          Yes I did do that, didn't I. With MRQ and VBR - 2 Pass it took 5 hours.

                          With enabled it took 30 hours (didn't try again, it crashed at 96% so an estimate).

                           

                          What I mean to ask is, slow performance is only due to hardware bottleneck or could it be software also? If yes how do I find what is the weak link?

                          Or, I am just simply thinking, if all my hardware is utilized only 30% is Neat Video a poorly written software not to make most it?


                          Further, as you can see from the screenshots. When I used Premiere Pro GPU load was 0-1% but when I used Media Encoder it was using it properly. How and Why? Any links to a help document will be appreciated.

                          • 10. Re: Help me identify problem for slow export on a powerful machine
                            JSS1138 CommunityMVP

                            It's very possible for third-party plug-ins to slow things down.  Any 'denoising' effect will do that.  MB Looks is also notorious for being really slow to render.