For your processor, in order for AE to work correctly, you'll have to turn off the multithreading in windows control panel.
The in AE, as you only have 3 gb of ram, I would enable multiprocessing and would use only 2 cores for AE (set the leave cores for other processors to 6 as Nehelem is 8 core I think).
Then i'll let 1gb of ram for you other apps. You'll then have 2 processors using each 2.5gb of ram for AE.
By the way, AE is a 32bit app, so on 1 process (without multiprocessing) it won't be able to use more than 4gb of ram, that's why it will not show on the splash screen at start more the 4gb of ram.
As Sébastien explained, 32bit app = 4GB RAM usage at most. As for your Vector Paint error - have you tried it with OpenGL turned off? We know that there is an issue with this effect when using the "Live Paint" mode (using the Shift key to record strokes), but otehr than that the effect seems to work. If it hinges on OpenGL, it would be specific to your system. Please try in Adpative Resolution mode with no OpenGL and report your findings.
Thanks for answers.
But still problem persists....
i turned off opengl but still i got the crash, but i recognised something,
when i start painting it, it DOES NOT GİVE the error IMMEDIATELY i mean i got the error after a while.
it is not about start painting i think, it is about the effect itself, when i apply the effect and a little time goes by i got the crash.
i couldnt find how to close multithreading and got the information that it can not be closed maybe i am wrong but i just couldnt find the options,
except this i tried your settings but video previews got a little bit slower then i turned into cpu usage 4 and per core usage 1 gb ram and i left 1 gb ram for other applications and i clicked "opengl always open" not interactive, it is a little better
Again thanks for your all answers.
Have a good day