System Spec's : 2.66Ghz 8-core (16 threads) Mac Pro 4,1 with 24GB RAM (4x2GB + 4x4GB)
Composition : 6400 x 720 pixels, lots of shape layers, feathering / blurring, Trapcode Particular, Fractal Noise. NO source files or footage.
Only other application running was Firefox (light usage)
RAM for other apps RAM per core Cores used Speed 4 2 8 3.13 6 2 7 3.20 6 3 4 3.00 8 1 14 5.16 8 1.5 9 3.11 8 2 6 2.53 8 3 4 2.56 8 4 2 4.04 10 2 5 2.55
This pretty much confirms teh recommendations Todd posted. In this case a VERY small speed increase was gained by allocating 1GB less per core, probably due to the comp having no source footage. When I get time I'll run the same tests on a footage-heavy comp.
I ran some quick tests on an older project (one of the only I have without lens blur and therefore one of the only where mp is not disabled).
It's 720x540, no video, all animated still images. Some fairly hi-res (2-3K), others only a few hundred px. Several adjustment layers over the whole comp for overall texture effects that really slow the whole thing down a lot.
These are times it took to render one second of the project.
1:44 (recommended settings)
So in this case the recommended settings actually delivered the worst performance by far.
I was confused about one thing in the explaination, when you mentioned setting aside all that ram for other applications, to me it sounded a little like that 6GB of ram is actually available to the foreground process of AE but not the background processes. Is this the case or is it really totally unavailable to AE?
BTW this is a 8-core Mac Pro 2.8 ghz, 16 gb ram, OS 10.6.4
I've been playing with multiprocessor settings on a number of computers, and I'm always noticing the same problem: when I try settings that seem reasonable, "Actual CPUs that will be used" drops to zero.
For instance, right this very moment: I have 12gb of RAM, 3gb reserved for other applications and 2gb allocated for background CPUs. If I try upping the allocation to 3, "Actual CPUs that will be used" now says 0. Which I presume means multiprocessing is actually disabled. Unless I'm misunderstanding what that means?
> I have 12gb of RAM, 3gb reserved for other applications and 2gb allocated for background CPUs. If I try upping the allocation to 3, "Actual CPUs that will be used" now says 0. Which I presume means multiprocessing is actually disabled. Unless I'm misunderstanding what that means?
12GB - 3GB reserved for other applications leaves 9GB. Setting minimum allocation per background process to 3GB also sets the minimum allocation for the foreground process to 1.2*3GB=3.6GB. So, 9GB - 3.6GB = 5.4GB. That only leaves enough RAM for one background process at 3GB. Keep in mind that if the background processes are rendering, then the foreground process is not. There's no point in starting up only one background process; so, in this case no background processes are started. Hence, the result that you see.
If you were to allocate 2.7GB as the minimum per process, then there'd be enough RAM for two background processes.
Note that you can set the allocation to whatever you want; you don't have to just use the values in the menu. See my comment at the bottom of this page.
Does that mean that the recommendation for 12gb is to not use multiprocessing?
I know "quick tests" in AE usually don't mean a thing, since each project renders differently, but I outputted a second from the project I'm working on -- some Liquify, lots of Particular, tons of precomps. With 3gb for other apps / 2gb per CPU (my go-to settings), I got a render time of 1:44. Rerendering the same frames with 4gb / 3gb, which disables multiprocessing, took 4:28.
So, I'm going to go out on a limb and say the 4gb / 3gb recommendation seems... off.
All layers are 1920 x 1080. 20 layers total, about half of those are PSD layers (all from the same PSD), the rest just solids. No footage.
Seven instances of Frischluft (Out Of Focus), one instance of RSMB, one instance of Optical Flares, one Liquify. Two instances of Particular that are really heavy -- 500 particles / sec with 1500 particules / sec aux system.
PSD layers were from an 8-bit file. Test render was a 16-bit TIFF sequence, 51 frames long.
Rendering on 3 cores (2gb / core) takes about 1/3 the time it takes to render with multiprocessing off (setting 3gb / core). Which is kind of what you'd expect.
Hopefully this helps.
ok, just ran some test here with a small project that includes:
6 precomps of large images turned into 3d buildings, a couple of lights, couple of adjustment layers, camera with slow pans and moves, 1080p24 sized.
rendered to default best settings and lossless.
machine: early 2008 8-core 2.8GHz, 16GB matched ram, rendering to 3x750GB Raid 0.
numbers mean this:(ie: 4-2-4.mov, 4GB for other apps, 2GB per BG proc, 4 cores used.
3-1-8.mov: 1min, 16sec
3-1.5-7.mov: 1min, 4sec
3-2-5.mov: 1min, 4sec
3-3-3.mov: 1min, 11sec
4-1-8.mov: 1min, 15sec
4-1.5-6.mov: 1min, 4sec
4-2-4.mov: 1min, 6sec
4-3-2.mov: 1min, 25sec
6-1-6.mov: 1min, 8sec
6-1.5-5.mov: 1min, 4sec
6-2-3.mov: 1min, 11sec
6-3-2.mov: 1min, 22sec
NUCLEO PRO 3:
One note from our test set-up:
Our engineers do custom installs they had to check on drivers. long and short, be sure the CUDA driver is installed (properly).
That combined with the recommended settings ensures a real performance boost. Thanks for this Todd it was a mystery in CS4!
We're still working on combinations and continuing with plug-in testing which is where we see huge differences. We'll post what we find.
Someone asked if the "Ram reserved for other applications" included the AE foreground process. It doesn't. It's possible the recommended settings in this thread are overly conservative, but it's been established that the settings we shipped with were too aggressive. The OS, especially OSX, needs quite a bit of memory headroom. Being too aggressive with this setting can result in swapping to disk unnecessarily which takes a very long time.
I agree that the 12GB settings are not ideal. Splitting the difference between the 8 and 16GB settings would be better. Off the top of my head, I'd say your "goto" settings(3GB reserved, 2GB per core) sound good.
I know our users want AE to use all of your cores, but try and think beyond "use all my cores" into "help me work as fast as possible". If you don't provide enough memory to your background processes, they'll be very inefficient and slower than if you used fewer cores. It's only "wasting a quadcore" when RAM AND CPU is available and unused.
Thanks for all the feedback. This is helpful to us.
Thanks for all the good feedback. The results reported are similar to those we are seeing in-house.
After Effects is a deep product that gets exposed to many workflows, from web animation to IMAX film compositing. Because of this broad exposure, there is no one ideal setting for all rendering. The proposed memory preference settings are just a starting point with the goal of a better out of the box experience when the user first turns on Render Multiple Frames Simultaneously multiprocessing. These settings are what the user would experience with a fresh set of preferences.
AE CS5’s current memory settings are the minimums necessary for Render Multiple Frames Simultaneously multiprocessing. We are thinking of raising the default preference settings to those listed on Todd’s blog ( http://bit.ly/ckIriV ) so that more users get satisfactory results without having to first tweak the memory and Render Multiple Frames Simultaneously multiprocessing settings. They are not meant to be a solution for every type of project. If one is working in NTSC resolution, lower settings might be acceptable. If one is working with RED footage in complex compositions, higher settings will be necessary for best performance.
RAM reserved for other applications:
By reserving more memory for other applications there is also more memory reserved for the operating system buffers. This is beneficial because it stops the OS from swapping to disk during memory intensive tasks. If you find that over time as you work with a project that interactions are slowing down and taking much longer than normal, it could be that the OS is swapping to disk. Look to see if the hard drive light is flashing almost constantly. On the Mac, open the Activity Monitor window (in the Utilities folder) and click on the System Memory tab. If the free memory is very low and the Swap used is high (over 500MB) and rising with each frame rendered, the OS is swapping. To alleviate this condition, raise the reserve memory setting in After Effects, save the project and restart the app. On the Mac, it may also help to restart the OS.
RAM allocation per background CPU:
We are also proposing raising the default background CPU setting from .75GB/ background CPU to 1.5GB / background CPU. The testing for these settings was done with a baseline of 1.5GB/ background CPU for Render Multiple Frames Simultaneously multiprocessing. This combination of reserving more memory on systems with generous amounts of RAM + higher background CPU memory allocations achieved successful, fast renders over a variety of real world HDV and HD projects.
Managing memory for Render Multiple Frames Simultaneously multiprocessing is complex. Think of each background instance as a mini After Effects. There are definitely projects that are going to require more memory per background CPU than others. For example, if the project will not render well with Render Multiple Frames Simultaneously multiprocessing off on a 2GB machine, it’s not going to render well with 2GB allocated to the background processes either. If you find this is the case, raise the background CPU allocation to a level that achieves the best result. In general we have found that web and NTSC work usually can be done with 1GB/ background CPU. HDV and HD work will usually need from 1.5 to 2GB/background CPU. Higher resolution work may need 3 or even 4GB/background CPU.
Please keep trying these proposed default settings with your projects and let us know how they work for you. Most importantly we need to know if they help reduce application slowdowns due to memory swapping, over the course of a day's work. That is the primary goal for the new defaults.
Since AE is capable of sending crash reports to you guys, why not build in the ability to send the essential render information? Size of comp, no. of layers, no. + data size of source files, render settings, system specs and render time. Make it optional and completely secure, confidential, anonymous etc etc
This is very helpful Aaron. It could be that you get fine results with just 1.5GB/ bkgnd process. If you have time, please up the reserved memory to 4GB with the BG cpu set to 1.5GB and try that for a while. Let us know how that works both for performance and for a working, all day setting that doesn't get starve the OS when you have been working for hours with the app.
> I should also mention that I’m running 64-bit Win 7, and that I did not have “Use Open GL Render” checked (although would I/Should I see a performance boost with this box checked in the render settings?
No. Don't use the OpenGL renderer for anything that you care about the appearance of. It is a preview renderer that speeds some things up at the expense of the appearance of the image. And it often causes other problems.
Anyone else have this problem?
Regardless of how much I tell AE to "Leave for other applications," it never fails to use up every drop of RAM until I'm left with a stuttering desktop and 17MB +/- of RAM left. This happened before in CS4 but I used Nucleo to get around it (although that sometimes still gave me problems). Recently switched to a brand new MacPro 12 core with 26GB of RAM. I launch AE and open my file (still 19GB RAM free at this point) and hit render. Mid-way through it drops way down to __MB. Other times, I'll work for a while, hit render and walk away. Come back when the render should be finished and it is taking twice as long. At that point, I need to quit out and restart AE and do a new render, then it is back to normal. Not sure what's up as I never had any of these issues back in CS2 or before but once I went up to CS4/CS5 it has never worked right.
Similar problem is during RAM previews of larger files. Hit RAM preview and it takes forever "loading projects" in the background. I read that it supposed to only happen the first time you RAM preview but it happens to me every time until I render something. If I turn off Multiprocessing, that immediately goes away. When I view the Activity Monitor, the individual render engines "aeselflink" are all open during the "loading projects" vortex so I have no idea why it continues to do it.
The most recent settings when I had the problem listed above with 19GB RAM free before launching AE.
RAM reserved for other apps: 11GB
CPUs reserved for other apps: 4
RAM per CPU: 2GB
Actual CPUs used: 7
Of course sometimes this happens while I'm working in other apps, but sometimes the only things that are open are Mail and Safari. I mean, I'm working with 26GB of RAM here! Shouldn't AE recognize how much RAM is available at launch and leave the amount listed in the prefs?
Am I the only one experiencing this?
Sounds wrong for sure. Are there any 3d part effects involved? Please submit a screenshot and a sample project (without footage) that has this issue to : email@example.com. We'll take a look and see what is happening.
It could be that this is happening in multiple situations, but we need specifics. For example, CS4 multiprocessing is quite different than CS5. There are many vast improvements in CS5 not the least being that it is a 64bit application able to address a much larger memory space.
Having concrete specific examples is the best way to address this issue and to see if developing solutions are effective. Please submit a project or projects illustrating this issue. Also include a screenshot of the Memory and Multiprocessing preferences panel on the system.
I doubled my RAM this morning from 12 to 24, so I decided to do another round of testing.
The comp was very heavy: DNX sequences being read from the network, lots of keying, Particular, Optical Flares, and Twixtor. The recommendations for 24 seem to be bang-on:
Quarter-res, Quicktime DNxHD
12gb RAM, fastest render: 9:30
24gb RAM, 3gb background, 2gb / instance, 7 cores used: 5:02
24gb RAM, 8gb background, 3gb / instance, 5 cores used: 2:15
Full-res, Quicktime Lossless
24gb RAM, 3gb background, 2gb / instance, 7 cores used: 17:10
24gb RAM, 8gb background, 3gb / instance, 5 cores used: 12:34I'm impressed. I really didn't expect the settings change to make such a difference, especially since I didn't have much luck with the recommended settings for 12gb.
12gb RAM, fastest render: 9:30
24gb RAM, 3gb background, 2gb / instance, 7 cores used: 5:02
24gb RAM, 8gb background, 3gb / instance, 5 cores used: 2:15
24gb RAM, 3gb background, 3gb / instance, 5 cores used: 6:01
Ouch. I set it back to 8gb and quickly did another render, just to see if there was something else slowing it down (like reading from the network), and I got a new render time of 1:48.
Is there anyway to allocate more than 3GB of RAM per CPU? I've noticed that after effects RIPS through the first part of my compositions, but eventually it gets hung up and slows way way down. Granted, this is probably a function of my composition as much as it is a part of my settings, but I was just curious.
FYI - This is when following the recommended settings for 24 GB of RAM (8GB reserved, 3GB resereved)... I just wonder if restricting the processors to 3GB doesn't allow them to reach their full potential for more complex compositions.
it does not help you anything.
Ae was AND is bulit on old code, forever an ever.
More RAM, faster Machines ... it won't help you anything. Performance will starve forever and ever.
Only thing i can say is to have patience. I know there is a lot of expectation upon the new product manager and i hope that he will take alle the pressure away and concentrate on a new level of competence and power because i believe he does have a great team around him which is waiting for a good leader, otherwise they won't follow
A recent tutorial on memory settings posted on AE's Facebook page had me poking around the Details tab in the memory & multiprocessing preferences. In the video, the presenter had 12 GB of RAM and set his RAM reserved for other applications to 3 and the RAM available for AE was 9. When he checked this out in the details tab, it showed the "Max Allowed Memory" for AE to be 9. I have 26GB of ram and regardless of whether I set the RAM reserved for other applications to 1GB or 25GB, the details tab tells me the "Max Allowed Memory" for AE is 18.4GB. This is definitely the reason why if I have more than one application open (god forbid I try that on a 12 core machine with 26GB) AE devours all RAM and leaves me with sometimes less than 10MB of RAM during renders and I can barely even check email.
One of our testers has been working on the issue that you reported, and here's what he has determined so far:
"I have reviewed this problem on a 16GB/8core Mac and on a 32GB/16core Mac. On both platforms After Effects is performing correctly for me. That is, if I increase the memory reserved for other apps, that change is reflected in the Memory Details. It should be noted that there is about a 6 second lag before the details updates with the new setting. This is by design. If Kenfai is not seeing this change on his machine, then something is wrong and we will need to investigate further.
I tried rendering kenfai’s bugfile project submitted to aebugs. On the 16GB Mac, I reserved 4GB for other apps and set the background cpus to 2GB (4 BG CPUs). The project rendered in 3m57s, without issue. No swapping or low memory state on the OS level was observed. I also tested this same project on the 32GB Mac with 18GB reserved, 2GB/BG CPU (5 BG CPUs). The project rendered in 1m51s with no issues.
What I suspect is that kenfai is seeing a Mac OS X bug that we have reported to Apple. When a lot of demand is placed on the memory system the OS will suddenly block off a chunk as “Inactive Memory”. On a machine with a lot of RAM installed the size of the buffer can be substantial, often 3-5GB. This is supposed to be a dynamic swap buffer that is available if an app calls for it. However, it may not release memory to AE. If so, AE starts using free memory. AE is not using more than it has been allocated, it’s just that the OS appears to be getting overprotected. The bug in the OS is that sometimes this memory is never released, even if all applications are closed. At that point the only ways to free memory are to restart the OS.
The user can clearly see what is happening by opening the Activity Monitor utility, and selecting the System Memory tab. The pie graph will show the inactive memory in blue. One can also watch the After Effects process to be sure that it is not exceeding its memory allocation.
Hope that helps him out"
Thank you for the update. Here's the thing, I've had this problem on my current machine (MacPro, 12 core, 26GB RAM, Snow Leopard) as well as my previous machine (MacPro, 8 core, 16GB RAM, Leopard) so I'm not quite sure this is an OSX problem. Regardless, when I go to my AE settings right now and set RAM reserved for other applications to 25GB, I see that RAM Available for AE is now set to 1GB which is correct. When I click the Details tab I see that Max Allowed Memory for AE is 18.4GB, not 1. This number never drops below 18.4 regardless of my settings and CPU allowance and I'm wondering if this is the problem. I do see that there is inactive RAM, etc but taking that in to consideration, if I see that there is only 17GB of free RAM to use, shouldn't I be able to tell AE to only use 9GB and still have 8GB free? That is never the case, AE always eats up every bit of RAM.
Can you try deleting these two preferences files on this Mac?
Restart the machine first. Then with no apps open, delete the After Effects 10.0 preference file ( Adobe After Effects 10.0-x64 Prefs.txt ) located in the /Users/username/Library/Preferences/Adobe/After\ Effects/10.0 folder.
Delete the memorybalancercs5.xml file located in the /Users/username/Library/Preferences/Adobe/dynamiclinkmanager folder.
Start the app and set the preferences.
I have a mid 2010 iMac, it was shipped with 8gb and I was eager to increase its render performance by adding the 16gb limit of ram. However, after installing all this ram, my results are disappointingly similar to before ! I have forced Snow Leopard to boot in 64 bit and I’ve played with all the recommended settings found here. I also ran my own tests, even tried switching of multiprocessing, but still saw no improvement whatsoever.
Stranger still, when I open up the activity monitor to look at the memory usage during a render, it never shows any significant exploitation of the ram, with almost 12 gb that stay desperately idle (whereas I've alway known mulitprocessing to be a real memory hog).
Am I missing something ?