I'm having trouble when using multiprocessing with aerender. When I'm rendering on a farm with multiple machines, we use the "Skip Existing Files" checkbox so multiple machines can work on a render at the same time. So normally when we render (without the multiprocessing turned on) it will quickly (0-0.1 seconds) skip over frames that have already been rendered, or are rendering on another machine.
PROGRESS: (Skipping 00607) (608): 0 Seconds
For some reason with multiprocessing turned on, this processing of skipping existing frames can take up to 20 seconds!! Often taking more time to skip the frame than actually render it.
PROGRESS: (Skipping 00607) (608): 18 Seconds
Anyway to fix this?
I know one alternative is to just not use the multiprocessing flag. However, I seem to get faster renders with it enabled on our 8 core machines, and the multiprocessing memory management has some benefits...
Normally under CS5 or CS5.5, aerender will grab every last bit of available RAM it can when rendering. This is not a good thing if we are background rendering on a workstation! However, with multiprocessing enabled, aerender uses the memory settings set for multiprocessing for all instances of after effects that are running on that machine. So if we have aerender running in the background with a user working on After Effects in the foreground. The total memory set in the After Effects multiprocessing preferences is shared among both processes. This makes it better as far as memory settings to render in the background while working in the foreground.
AERender used to properly work with the -mem-usage flag to limit the amount of memory available to that process, but under CS5 there is a bug where this flag has no effect...
Our machines are all Macs running snow leopard.
Any suggestions on either skipping frames quicker, or other alternatives to RAM issues?
Interesting note, I have a machine with 12GB of ram and 8 physical cores which renders a particular sequence with an average of 13 seconds per frame. I have another machine with 4GB of ram and 4 physical cores which on the same sequence renders at the same average of 13 seconds per frame.
When rendering both machines have aerender maxing out the memory usage. So the question is, why does the machine with 8 cores use 8GB more RAM than the 4 core, and still render at the same speed? Or, how do I limit the memory usage on the 8 core to 4GB since it's going to render at the same speed anyway.
Any help would be greatly appreciated! Been struggling with this for a while now
> Anyway to fix this?
There is currently a bug/limitation that makes Skip Existing Frames and Render Mutliple Frames Simultaneously multiprocessing incompatible.