You may be using a specific plug-in/ effect/ feature that continually requests temporal information. Could be by design, could be intentional. also, naturally, expressions will get in the way of cacheing operations many times...
Yup! Expressions is what's killing it. I have several LinesCreator 3D lines which uses copious amount of code to do its magic.
Shake 'n Bake those expressions!
If you post the expressions that are causing the slowdown to the Expressions forum here we may be able to help. Sometimes it's easy to get a loop going that has to go back and re-calculate every frame for the next one. These are render killers.