Well, different question: Is the Rotobrush stuff axctually frozen? Even if it is not actualyl propagating, it can be slow if not the whole solution has been baked... Other than that I'd simply turn off any MP. this might work better than wasting memory on background processes that you don't seem to be using, anyways...
I may be missing something here, but it looks like you're running a 12-core machine (dual 6 core), right? I'm not sure why AE is seeing 24 cores. Is there some weird hyperthreading going on? Maybe it's late for me and my brain isn't working properly though. Maybe try resetting the prefs? Disable multi-processing and OpenGL and see if that improves performance?
Edit: Lutz beats me out of the gate, again.
It was indeed not baked, thanks for the tip. Never to old to learn something new. I'll be the first to tell you Rotobrush is a great tool, but I do not know it as well as I should. Thanks for the schoolin'