You are using way too many cores.... Even for "light" comps, 8 cores with only 12GB is not really a good way too work. You need a lot more RAM. Set the render instances to 3 or 4 at most in your current environment.
So it sounds like a lot of cores will be redundant.
Would it be good idea to throw more RAM in then? I mean cost effeciant?
How much RAM per core should I consider, and how many of my cores should I aim to use?
> How much RAM per core should I consider, and how many of my cores should I aim to use?
It depends on what kind of work you're doing and whether the rest of your system is powerful enough to not introduce other bottlenecks.
A good rule of thumb is to have 3-4GB installed per core, and then only assign 2/3 to 3/4 of that to After Effects. I have 24GB installed in my hyperthreaded quad-core computer, which has 8 processors counting the virtual processors created by hyperthreading. I leave 2 processors and 6GB for other applications, leaving 6 processors and 18GB for After Effects. This works well for me when I'm working on 2K RED frames. I move to using only 4 processors when I'm using 4K frames.
Follow the link that Szalam gave. It has our current recommendations for starting points---but they're still just starting points.
BTW, regarding the bottlenecks that I mentioned at the top of this response: If you've got a lot of processors, you need to make sure that you've got fast hard disks that can feed them with input (source files) and offload their output (rendered and encoded frames). I see way too many people with screamingly powerful computers that are bogged down trying to push and pull data through tiny I/O pipes.