Didn't think this was possible with 96Gb RAM and 8GB RAM per CPU allocated.
Why not? Depending on what footage items and effects you use, you could easily gobble up all that memory. Impossibole to tell, since you lost yourself in explaining the technicalities without providing any info as to the nature of the projects - what footage formats and resolutions, what effects and all that. Some effects just don't work reliably with MP and likewise, your footage I/O could simply cause delays and tiemouts that bomb your background instances. In fact you might be better off using network rendering or using the BGRender script hack...
There are almost no effects used. I am aware of the effects that are not compatible with MP and we are not using them in any of our projects, For the most part, we take assets from the clients and repurpose them into AE to conform to our layout - mostly PSD files (usually go through the client stuff, clean it up, flatten layer effects, rename layer names and rasterize fonts, etc), Quicktime movies (Animation compression) and Illustrator files. The most intense that we usually get is using something like a FAST BLUR effect.
Should I simply abandon trying to use Multiple Frame rendering with all that RAM installed? Again, I thought this was the whole point, especially since we are rendering TGA sequences - my assumption was that rendering 8-10 TGA files simultaneously would be faster than 1 or 2 for a 900 frame sequence. I'd like to think that using multiple frame rendering is possible without it failing every time. I have opened up and tested past projects on our older machines that produced the content on this new machine and all have failed within the first 100 frames of rendering with MP switched on.
Currently, out of 96GB RAM, I have 8 GB reserved for other applications. Of the 24 CPUs available to AE, 18 are reserved for other applications (6 CPUs to be used by AE) and for the RAM allocation per CPU, I have it set to 8GB RAM per CPU. Not sure what else to try or if in fact my assumption of using MP rendering would be faster than not using it with 96 GB RAM in the box is false
Ok, I was able to get a couple of our past projects to use MP without crashing as test renders. However, it seems whenever there is a QuickTime movie used in the project as source footage, the MP rendering fails. This occurs with all flavors of QuickTime movies tested so far. The latest render had a Quicktime .MOV file that was 66.1MB, H.264 at 1920 x 1080 as well as a bunch of TrapCode effects (which rendered fine using MP). Once the render reached the .MOV file, the MP render failed (mostly produced 0K frames and then ultimately crashed AE)
After Effects warning: Sequence file #808 (zero based) failed to render (26 :: 142)
After Effects error: Error (4) reading frame from file “\\VS-DATA-01\Production\Signs\2012 Projects\04-25 AE 2xRes Test\AE\(Footage)\_Assets\photos-main\Puss in Boots_0012.mov”. (86 :: 2)
Another example of this occurred when there was a Quicktime movie in the project that was using Animation Compression codec, at 1920 x 1080 and was 155MB
Output To: C:\ec\NFLRenderTest\Top2\TOP_[####].tga
There is an error in background rendering so switching to foreground rendering after 255 frames completed out of total 900 frames. (26 :: 142)
After Effects error: Error (4) reading frame from file “C:\ec\NFLRenderTest\NFLRenderTest folder\(Footage)\~Work in Here\MOVs\01-iPAD.mov”. (86 :: 2)
Is there an overall obstacle to using QuickTime movies as sources when trying to output TGA sequences using Multiple Frame rendering? It seems this is consistent
From the error message, it would appear to be the nature of the footage. I know you said you used all flavors of QT codecs, but did any of them happen to be in the PNG, ProRes or Animation codecs?
The reasons I ask: any kind of codec with interframe compression like H.264 throw pre-CS5 versions of AE for a loop, and strange things happen. Then there's my inbred suspicion on interframe-compressed codecs...
Thanks for the reply Dave. From the projects cited above, one had a QuickTime with Animation codec and the other had H.264. I think we are getting close to honing-in on the issues we are having as it seems that anything with a .MOV as a source file seems to throw AE MP rending for a loop. Currently using AE 5.5. Conversely, MP plays very nicely with projects that do not include a QuickTime file as a source; I have been doing extensive testing using the following settings; 16GB/CPU - 4 CPUs used, 12GB/CPU - 6 CPUs used, 10GB/CPU - 8 CPUs used, 8GB/CPU - 10 CPUs used and 6GB/CPU - 12 CPUs used.
I guess I still have a few questions remaining in trying to troubleshoot the issue at hand:
- By any chance are there any MP improvements in CS 6 that may alleviate the issue we are experiencing?
- I am wondering if I am hitting a bottleneck getting the info (.MOVs) into RAM - to be more specific regarding my setup, in addition to my previous post - My machine currently has two 250GB SAS drives @ 10K RPM that are RAID 0. Disk cache is set to the default settings. We are considering additional drives - thinking one for OS & Applications, 1 for source files and one for output of the image sequences. Any insight to drives, where to set the cache, types of drives (SSD drives fall out of budget for the machine I have - already priced and axed. I think we are considering two more 10K SAS drives as an option)?
- This one is a longshot, but I thought I saw a crazy post somewhere in my research of this issue saying to get rid of QuickTime. I was always under the assumption that QuickTime was a vital prerequisite for AE. Any truth/validity in getting rid of QuickTIme?
QT X has a nasty reputation for flaky performance. I don't even know if available for Windows, but if it is and you have it, uninstall & go with the latest version of QT 7 for Windows.
Here's another basic thing: don't export, use the AE Render Queue. I don't know if that variable has been mentioned yet.
Using latest version of QuickTime (7.7.2). We don't use export, we always use the render queue as we need to render 2 TGA sequences for TOP & BOTTOM comps.
We are running into essentially the very same issue with our new workstations using dual E5-2687's with 128GB of ram (4GB per logical thread, 8GB per core) dealing with quicktime files of over 2560x1080 in CS6. I was able to pinpoint quicktime as a contributing factor since the problem disappears if you convert those QT files to AVI uncompressed or image sequences. However anything that has to go through the 32bit QTserver thread (all quicktime files) run into this issue. I think it may be related to the 32Bit hack that After Effects uses to keep Quicktime working with the new 64bit render engine in After Effects CS5+
We are currently testing changing our workflow to eliminate quicktime from it, as there does not seem to be any progress from either Adobe or Quicktime in addressing this problem. (it has persisted from CS5 through 5.5 and 6, and several changes to quicktime.)
Ah, I should have posted an update after speaking with Todd Kopriva on a number of occaisions regarding our custom use of AE. The 32bit QTServ is indeed the culprit. Once I pre-rendered the QT movies out as JPGs and reimported, the issues went away. Too bad QT will never be a 64-bit application on the Windows platform