I'm using Primatte Keyer to key some (admittedly large) R3D footage, and eventually the preview goes black and I get an error message saying there is not enough memory to allocate. I check Acitivity Monitor (I'm on a mac) and AE is only utilizing 8GB of memory. I have 32GB installed, and 26GB allocated to Adobe Programs in the preferences. No other Adobe programs are open. Is the problem in Primatte, or is there some glitch I'm unaware of that AE underutilizes available RAM?
The memory allocation is split across threads/ cores and single-threaded plug-ins may never allow to exhaust these limits. So yes, it's probably a Primatte thing.
Also if you are on a Windows machine some versions (Windows 7 Home for example) have limits on how much RAM they will utilize and that is completely independent of how much you can install in the computer.