9 Replies Latest reply on Nov 16, 2012 1:30 PM by Dondi_

    Render Multiple Frames Headache

    Dondi_

      I hope I could get some guidance on optimum usage of Render Mutiple Frames as I have been hitting a wall with little-to-no success so far. I have read all the Adobe suggestions (thanks Todd Kopriva) and I still get failed renders when trying to render multiple frames. I am the guinea pig for the hardware setup for the department. Currently, the hardware is a Lenovo ThinkStation C20X, dual Xeon 6-core 2.4 GHz processors (24 available to AE with hyper-threading), 96 Gb RAM (yes, ninety-six gigs), Win 7 Pro 64-bit, nVidia QUADRO 2000D, AE CS5.5. Our department currently works in an unusual AE project file that is used for display advertising on an unusually-shaped gigantic digital screen in Times Square. Currently, the working comp is 2517 x 4478 which is then nested into two comps for output of TGA sequences that are: TOP = 440 x 3694 and, BOTTOM = 2517 x 554. Most of the working comp space is deadspace as the layout of the signs is comparable to a large upside-down capital letter 'T'. We are currently in transition as the signs are hitting their life cycle and we are refreshing them with a finer pitch-count - in essence, we are doubling the resolution. The new working comp is 5034 x 8956 that is nested into our output comps which are: TOP = 852 x 7386 and BOTTOM = 5034 x 1098. For the most part we do not use "crazy" effects, etc. I have tried rendering multiple frames and have yet to get TOP & BOTTOM to both render without either one or both of them failing using various trial-and-error settings in the preferences; I have limited the # of cores (have yet to try to exceed 8 simultaneous CPUs). I edited the preferences file to allow me to select more RAM per CPU (currently, my pull-down has 4, 5, 6 & 8 Gb RAM per CPU available for me to choose from). Maybe I am going under the false assumption that filling the machine with 96 RAM would allow me to effectively use more CPUs in rendering TGA sequences. I thought that was the whole point. I usually get errors like the following:

       

      Output To: C:\Sonic RenderTest\Top\TOP_[####].tga

       

         There is an error in background rendering so switching to foreground rendering after 41 frames completed out of total 900 frames. (26  ::  142)

      After Effects warning: A frame failed to render while using Render Multiple Frames Simultaneously. Allocating more memory to the background processes in Memory & Multiprocessing Preferences may fix this problem. (26  ::  142)

       

      Didn't think this was possible with 96Gb RAM and 8GB RAM per CPU allocated. Any guidance here is greatly appreciated as I am on the hook with the rest of the department and with my boss as I convinced him that 96 GB RAM would make our lives easier.

        • 1. Re: Render Multiple Frames Headache
          Mylenium Most Valuable Participant

          Didn't think this was possible with 96Gb RAM and 8GB RAM per CPU allocated.

           

          Why not? Depending on what footage items and effects you use, you could easily gobble up all that memory. Impossibole to tell, since you lost yourself in explaining the technicalities without providing any info as to the nature of the projects - what footage formats and resolutions, what effects and all that. Some effects just don't work reliably with MP and likewise, your footage I/O could simply cause delays and tiemouts that bomb your background instances. In fact you might be better off using network rendering or using the BGRender script hack...

           

          Mylenium

          • 2. Re: Render Multiple Frames Headache
            Dondi_ Level 1

            There are almost no effects used. I am aware of the effects that are not compatible with MP and we are not using them in any of our projects, For the most part, we take assets from the clients and repurpose them into AE to conform to our layout - mostly PSD files (usually go through the client stuff, clean it up, flatten layer effects, rename layer names and rasterize fonts, etc), Quicktime movies (Animation compression) and Illustrator files. The most intense that we usually get is using something like a FAST BLUR effect.

             

            Should I simply abandon trying to use Multiple Frame rendering with all that RAM installed? Again, I thought this was the whole point, especially since we are rendering TGA sequences - my assumption was that rendering 8-10 TGA files simultaneously would be faster than 1 or 2 for a 900 frame sequence. I'd like to think that using multiple frame rendering is possible without it failing every time. I have opened up and tested past projects on our older machines that produced the content on this new machine and all have failed within the first 100 frames of rendering with MP switched on.

             

            Currently, out of 96GB RAM, I have 8 GB reserved for other applications. Of the 24 CPUs available to AE, 18 are reserved for other applications (6 CPUs to be used by AE) and for the RAM allocation per CPU, I have it set to 8GB RAM per CPU. Not sure what else to try or if in fact my assumption of using MP rendering would be faster than not using it with 96 GB RAM in the box is false

            • 3. Re: Render Multiple Frames Headache
              Dondi_ Level 1

              Ok, I was able to get a couple of our past projects to use MP without crashing as test renders. However, it seems whenever there is a QuickTime movie used in the project as source footage, the MP rendering fails. This occurs with all flavors of QuickTime movies tested so far. The latest render had a Quicktime .MOV file that was 66.1MB, H.264 at 1920 x 1080 as well as a bunch of TrapCode effects (which rendered fine using MP). Once the render reached the .MOV file, the MP render failed (mostly produced 0K frames and then ultimately crashed AE)

               

              After Effects warning: Sequence file #808 (zero based) failed to render (26  ::  142)

              After Effects error: Error (4) reading frame from file “\\VS-DATA-01\Production\Signs\2012 Projects\04-25 AE 2xRes Test\AE\(Footage)\_Assets\photos-main\Puss in Boots_0012.mov”. (86  ::  2)

               

              Another example of this occurred when there was a Quicktime movie in the project that was using Animation Compression codec, at 1920 x 1080 and was 155MB

               

                 Output To: C:\ec\NFLRenderTest\Top2\TOP_[####].tga

               

                 There is an error in background rendering so switching to foreground rendering after 255 frames completed out of total 900 frames. (26  ::  142)

              After Effects error: Error (4) reading frame from file “C:\ec\NFLRenderTest\NFLRenderTest folder\(Footage)\~Work in Here\MOVs\01-iPAD.mov”. (86  ::  2)

               

              Is there an overall obstacle to using QuickTime movies as sources when trying to output TGA sequences using Multiple Frame rendering? It seems this is consistent

              • 4. Re: Render Multiple Frames Headache
                Dave LaRonde Level 6

                From the error message, it would appear to be the nature of the footage.  I know you said you used all flavors of QT codecs, but did any of them happen to be in the PNG, ProRes or Animation codecs?

                 

                The reasons I ask: any kind of codec with interframe compression like H.264 throw pre-CS5 versions of AE for a loop, and strange things happen.  Then there's my inbred suspicion on interframe-compressed codecs...

                • 5. Re: Render Multiple Frames Headache
                  Dondi_ Level 1

                  Thanks for the reply Dave. From the projects cited above, one had a QuickTime with Animation codec and the other had H.264. I think we are getting close to honing-in on the issues we are having as it seems that anything with a .MOV as a source file seems to throw AE MP rending for a loop. Currently using AE 5.5. Conversely, MP plays very nicely with projects that do not include a QuickTime file as a source; I have been doing extensive testing using the following settings; 16GB/CPU - 4 CPUs used, 12GB/CPU - 6 CPUs used, 10GB/CPU - 8 CPUs used, 8GB/CPU - 10 CPUs used and 6GB/CPU - 12 CPUs used.

                   

                  I guess I still have a few questions remaining in trying to troubleshoot the issue at hand:

                   

                  - By any chance are there any MP improvements in CS 6 that may alleviate the issue we are experiencing?

                  - I am wondering if I am hitting a bottleneck getting the info (.MOVs) into RAM - to be more specific regarding my setup, in addition to my previous post - My machine currently has two 250GB SAS drives @ 10K RPM that are RAID 0. Disk cache is set to the default settings. We are considering additional drives - thinking one for OS & Applications, 1 for source files and one for output of the image sequences. Any insight to drives, where to set the cache, types of drives (SSD drives fall out of budget for the machine I have - already priced and axed. I think we are considering two more 10K SAS drives as an option)?

                  - This one is a longshot, but I thought I saw a crazy post somewhere in my research of this issue saying to get rid of QuickTime. I was always under the assumption that QuickTime was a vital prerequisite for AE. Any truth/validity in getting rid of QuickTIme?

                  • 6. Re: Render Multiple Frames Headache
                    Dave LaRonde Level 6

                    QT X has a nasty reputation for flaky performance.  I don't even know if available for Windows, but if it is and you have it, uninstall & go with the latest version of QT 7 for Windows.

                     

                    Here's another basic thing: don't export, use the AE Render Queue.  I don't know if that variable has been mentioned yet.

                    • 7. Re: Render Multiple Frames Headache
                      Dondi_ Level 1

                      Using latest version of QuickTime (7.7.2). We don't use export, we always use the render queue as we need to render 2 TGA sequences for TOP & BOTTOM comps.

                      • 8. Re: Render Multiple Frames Headache
                        ANC_Sports2

                        We are running into essentially the very same issue with our new workstations using dual E5-2687's with 128GB of ram (4GB per logical thread, 8GB per core) dealing with quicktime files of over 2560x1080 in CS6. I was able to pinpoint quicktime as a contributing factor since the problem disappears if you convert those QT files to AVI uncompressed or image sequences. However anything that has to go through the 32bit QTserver thread (all quicktime files) run into this issue. I think it may be related to the 32Bit hack that After Effects uses to keep Quicktime working with the new 64bit render engine in After Effects CS5+

                         

                        We are currently testing changing our workflow to eliminate quicktime from it, as there does not seem to be any progress from either Adobe or Quicktime in addressing this problem. (it has persisted from CS5 through 5.5 and 6, and several changes to quicktime.)

                        • 9. Re: Render Multiple Frames Headache
                          Dondi_ Level 1

                          Ah, I should have posted an update after speaking with Todd Kopriva on a number of occaisions regarding our custom use of AE. The 32bit QTServ is indeed the culprit. Once I pre-rendered the QT movies out as JPGs and reimported, the issues went away. Too bad QT will never be a 64-bit application on the Windows platform