Save the project and restart the computer to purge most of the stuff AE stashes here and there.
You're going to start a binary test to attempt to isolate the cause.
Open a new 2kx2k comp in AE and put your photos in it, one at a time. Scale the picture to fit. Export a still image. Import the file back into your project.
Make a copy of your problem comp and replace your photos with the new, smaller versions. What happens now?
Turn off all effects, one at a time, and evaluate the situation with each change.
Turn off each layer, one at a time, and evaluate the situation.
Blurs on huge files like 4k images take a long time to process. You can get around that by applying the blur to the image in a pre comp and exporting it as a still or as a movie if the effects change over time.
I'll have to try that out when I have time. That's not an option for me at the moment since as usual, this comes up while I'm meeting a deadline. In a way, you've answered my question as it seems that it's not a common thing with the new version.
I'm having the same issue (AE 2014 CC)
Compared to Premiere Pro the speed of this program is horrendous, random pauses aside it feels like it was poorly optimized.
I'm amazed at the incredible projects people manage to create in After Effects - they obviously have more patience than me.
I just updated everything today.
Popped open a couple of recent projects, converted them to the new format and ran a few quick tests to see what has changed or if there are any new processing issues.
AE works as well as it ever has on my MacPro cylinder running Mavericks.
Sorry, but I cannot confirm your unfavorable performance issues.