Nor do I understand why it isn't purging unused memory as it moves along in the render.
Why should it? AE never releases memory until the very last moment due to its "lazy caching" approach. Likewise, recent versions of Cinema 4D do some RAM caching for the output and retain multiple frames up to a certain point. The rest is unclear. I'm reading a lot of vague stuff without any actual screenshots and details. For all I know, this might have nothing to do with the C4D layers at all but could e.g. be an issue in the AE track matte usage.
Why should AE purge RAM while it renders? Because frame 240 doesn't rely on the outcome of frame 239, much less frame 1, in order to render. It's an incredibly inefficient use of RAM. I'm not saying AE is to blame—it could be Cineware that's the problem—but I've seen far more complex projects render in Maya on much less RAM.
I cannot provide screenshots for IP reasons, but I'm happy to provide more details about the project if it would help to troubleshoot this.
Basically, I've shot video of an instrument that has lasers running through it. I'm using Cinema4D to create the lasers and AE composites the lasers with the video. There are two C4D layers in AE. One consists entirely of parallel spot lights serving as the lasers. The track matte consists of geometry that represents that shape of the instrument that obscures the lasers at certain points in the image (i.e. if the laser goes through a tube, there is geometry for that tube).
For others using Cineware, are you finding similar render times between C4D and AE? Even if I drop the resolution of AE to a quarter, which is roughly the resolution that C4D Lite is rendering at, it still takes about 4x as long to render at C4D.
Are you still experiencing this performance issue, Ben? Please let us know. I want to make sure that this was addressed in the latest release.