15 Replies Latest reply on May 22, 2017 6:42 PM by hungaristani

    Randomly varying render times

    hungaristani

      I'm working on a 1hr 40mins multicam HD project, one composition per camera, each of which contains one pre-comped composition. When I put multiple compositions into the render queue, I get totally randomly changing rendering times, e.g. sometimes a composition takes 5 hrs, sometimes 10 hrs or even longer. No settings are changed between two renders and the modification I make to the comps are very minor and affect only some very short clips.

       

      I have the impression that it could be a RAM flushing issue or something like that, when I start a new render after a fresh system reboot (it's not enough to restart AE), the first comp in the cue always renders fast, then the subsequent ones become sluggish in most cases, but not always (which is really puzzling).

       

      Nothing else happens on the machine during rendering and all other circumstances (display refresh etc.) are exactly the same. Memory usage is usually around 66%, there's plenty of free disk space, nothing is maxed out, CC2017, Win 7 64 Ultimate.

       

      Has anyone noticed this? (Please, skip the links and hints about optimizing the render times, as this is clearly a different issue).

        • 1. Re: Randomly varying render times
          Mylenium Most Valuable Participant

          (Please, skip the links and hints about optimizing the render times, as this is clearly a different issue).

          Why shouldn't we? You may not like it, but you have told us literally nothing useful, meaning exact system info, exact render settings, comp settings, details about comp contents, cache settings, storage. I will indeed forego posting the same links the ump-teenth time in the interest of keeping peace and quiet, but really, your post is pretty useless. You have to be much more specific and precise rather than throw around generic info and complain about unpredictable render times.

           

          Mylenium

          • 2. Re: Randomly varying render times
            Horshack Level 4

            If restarting AE doesn't normalize the render times but rebooting the system does then that suggests there is something at the system level affecting the render times that is occurring/triggered by the first render you're performing. When the first render completes there is likely still dirty cache disk  data in memory corresponding to the file generated by AE - that data will be flushed lazily by the operating system. It's likely that the flushing of that data will slow down the initial stages of the second render; however, the size of the cache is limited by the size of your memory so it's unlikely the flushing of that data will take very long to complete and so even if this is occurring it shouldn't have the drastic impact you're seeing.

             

            So the first thing I would suggest is opening up the Windows task manager and click on the 'Performance' tab. Watch the graph for CPU, Memory, and Disk right as the first render following a reboot completes. Continue watching as the second render begins and see if you notice any significant difference in the utilization of the CPU, Memory, and Disk. Watch the progression of that second render; if it appears to be tracking much slower than the first (your symptom), then pause the render and look at the Performance monitor again to see what's happening - it should return to idle in terms of CPU and Disk usage.

            • 3. Re: Randomly varying render times
              Szalam Adobe Community Professional & MVP

              In addition to the things mentioned by Horshack, I'm curious about how you have your disk cache set up.

              • 4. Re: Randomly varying render times
                Dave LaRonde Level 6

                "I'm working on a 1hr 40mins multicam HD project, one composition per camera, each of which contains one pre-comped composition."

                 

                Um, why are you even using AE in the first place?  It seems to me Premiere Pro is the application for this kind of work.  You can color correct in it, and it's a LOT better for multicam work.

                 

                Unless, of course, there's something you haven't mentioned yet.....

                • 5. Re: Randomly varying render times
                  hungaristani Level 1

                  Unless, of course, there's something you haven't mentioned yet.....

                  Yes, there is...

                  I'm not using AE for multicam editing. The only reason why I mentioned that it's a long multicam project was to indicate that I have several (14) long (1hr 40mins) comps.

                   

                  Initially, I decided to apply the noise reduction to the raw material in AE before taking it to the multicam editor. The noise reduction (in NeatVideo for AE) is way simpler when I have to treat 14 cameras only (i.e. apply 14 noise profiles) rather than doing it on the finished movie with 2000 clips, where in many cases it's nearly impossible to recognize which clip was shot by which camera.

                  This requires longer render times at the beginning of the project (because all files are treated from beginning to the end, including parts which are not used in the final movie), but this extra render time pays off a thousand times when doing the editing and also makes future modifications a lot easier, because I don't have to worry about the noise reduction of the newly added or modified clips.

                   

                  Things got nastier when I learnt that for legal reasons there are a lot of clips where certain parts had to be masked out (and sometimes get replaced with other content). This involved a huge amount of rotoscoping, mask tracking etc. resulting in a much more complex project and dramatically increased render times.

                  At this point these randomly changing render times started to matter.

                  • 6. Re: Randomly varying render times
                    hungaristani Level 1

                    Pausing drops CPU usage back to near 0 figures, but keeps data in the memory.

                    Stopping a render manually has the same effect.

                    I don't know what successful completion does to CPU/RAM, but the machine is rendering right now, so I have to wait several hours to be able to check that.

                    Closing AE releases both CPU and RAM (but, as I mentioned, once rendering slows down, closing and relaunching AE doesn't help).

                    • 7. Re: Randomly varying render times
                      hungaristani Level 1

                      Mylenium  wrote

                       

                      (Please, skip the links and hints about optimizing the render times, as this is clearly a different issue).

                      Why shouldn't we? You may not like it, but you have told us literally nothing useful, meaning exact system info, exact render settings, comp settings, details about comp contents, cache settings, storage. I will indeed forego posting the same links the ump-teenth time in the interest of keeping peace and quiet, but really, your post is pretty useless. You have to be much more specific and precise rather than throw around generic info and complain about unpredictable render times.

                       

                      Mylenium

                      I guess you meant to say "why should we?" at the beginning, but anyhow, try to look at the bright side of my lame and useless post: it inspired you to create the single most valuable and insightful comment in the entire history of all forums.

                      • 8. Re: Randomly varying render times
                        hungaristani Level 1

                        Szalam  wrote

                         

                        In addition to the things mentioned by Horshack, I'm curious about how you have your disk cache set up.

                        The footage is read from a 2 SSD RAID0, the output is written to another 2 SSD RAID0, the disk cache is located on a separate HDD, the free disk space is currently 1.2TB, Max Disk Cache Size is 150GB. Media cache files are also located on the same HDD.

                         

                        When you mentioned disk cache, I remembered something that could have something to do with the issue. Occasionally (but not every time the render times increase) I got a warning about missing cache files. E.g.:

                         

                        After Effects warning: The file 'H:\AE Cache\Adobe After Effects Disk Cache - eNerd-PC01.noindex\Adobe\After Effects\14.1\Disk Cache - eNerd-PC01.noindex\25\25fa94d0-1b1b-375e-f307-4046010dfdb4.ADBE ICV.7308639.AAAAAHAIAA8=.aecache' could not be found.

                         

                        Every time I checked it, the disk was up and running, the referenced folder existed, but the file was really missing.

                        • 9. Re: Randomly varying render times
                          Dave LaRonde Level 6

                          Ugh.  Now, that's no fun.  I'd ascribe the things you had to do in the various clips to the varying render times.

                          And Neat Video, while a LOT faster than AE's noise removal, still takes its own sweet time.

                          • 10. Re: Randomly varying render times
                            Warren Heaton Adobe Community Professional

                            How about a different workflow?  Edit, lock picture, then do your noise removal only on the footage that you actually have in your cut.

                             

                            Also, NeatVideo offers their plug-in at a reduced price for other applications if you already own it.  I forget how much I paid for the AE version after having the PR version first, but it wasn't too much.

                            • 11. Re: Randomly varying render times
                              Ejik

                              во время работы в настройках кэш и носители кэша -сбросьте кэш не смотря на большое пространство на диске. Важно на каком движке вы работаете . В новых версиях прямо в композиции нажмите на неё и посмотрите то ,что Вам подходит , можно запустить движок от с4d намного быстрее пойдёт процесс . но я не знаю что вы делали в пре-композициях ,это может не отразить некоторые эффекты которые Вы применили?

                              Ejik

                              21.04.2017.

                              • 12. Re: Randomly varying render times
                                hungaristani Level 1

                                Ejik  wrote

                                 

                                во время работы в настройках кэш и носители кэша -сбросьте кэш не смотря на большое пространство на диске. Важно на каком движке вы работаете . В новых версиях прямо в композиции нажмите на неё и посмотрите то ,что Вам подходит , можно запустить движок от с4d намного быстрее пойдёт процесс . но я не знаю что вы делали в пре-композициях ,это может не отразить некоторые эффекты которые Вы применили?

                                Ejik

                                21.04.2017.

                                 

                                對不起?

                                • 13. Re: Randomly varying render times
                                  hungaristani Level 1

                                  https://forums.adobe.com/people/Warren+Heaton  wrote

                                   

                                  How about a different workflow?  Edit, lock picture, then do your noise removal only on the footage that you actually have in your cut.

                                   

                                  Also, NeatVideo offers their plug-in at a reduced price for other applications if you already own it.  I forget how much I paid for the AE version after having the PR version first, but it wasn't too much.

                                  I thought about that before starting the whole project. Actually, for compatibility reasons with other participants I have to do the multicam editing in Edius, but that shouldn't have made any difference in terms of the workflow. So, I bought NeatVideo for Edius in December, they charged my credit card, but their system screwed up something and I didn't get the licence -I can't remember what happened exactly, but it was something to do with my already existing account for the AE version-, then a couple of days later they sent me an email that their system doesn't allow to fix the problem manually, so they sent me a refund and instructed me to place a new order from a different account. This whole thing took too long and I had to start working on the project, so I started to render the files in AE. (BTW, the rates were not much lower for owners of other versions of NV).

                                   

                                  But it wouldn't have made any difference even if I'd bought the Edius version, because as I wrote earlier, it turned out in the meantime that I had to do a lot of rotoscoping and mask tracking etc., so I would have been forced to do the job in AE anyway and I would have had to do all these renders in there.

                                  • 14. Re: Randomly varying render times
                                    Kevin-Monahan Adobe Employee

                                    Hi hungaristani,

                                    Are you still facing this problem? If not, let us know how you solved it. If so, please let us know so we can assist you further.

                                     

                                    Thanks,

                                    Kevin

                                    • 15. Re: Randomly varying render times
                                      hungaristani Level 1

                                      Hi Kevin,

                                       

                                      yes, the problem still persists. My guess is that it has something to do with the warning about missing disk cache files I occasionally receive: e.g.:

                                      "After Effects warning: The file 'H:\AE Cache\Adobe After Effects Disk Cache - eNerd-PC01.noindex\Adobe\After Effects\14.1\Disk Cache - eNerd-PC01.noindex\25\25fa94d0-1b1b-375e-f307-4046010dfdb4.ADBE ICV.7308639.AAAAAHAIAA8=.aecache' could not be found."

                                      Every time I checked it, the disk was up and running, the referenced folder existed, but the file was really missing.

                                       

                                      Also, it seems that the problem occurs more often when I have several files in the render queue (cca. 100 min files each).

                                       

                                      So, I get the best results if I flush the cache before rendering and don't render more than 2 or 3 files in one go.