AE is first and foremost a "resolution independent compositing application", not a finishing program. That being so, there is no way to playback every arbitrary format and resolution in realtime nor is there a way to accelerate each and every operation in such a manner. A simple pan across an oversized photo qualifies for that, even if it may happen in a HD comp. That said, I'm sure one of those days AE will playback you some stuff straight off the bat. I'm just not sure how useful it would actually be to me...
The entire After Effects team is working on various performance features right now, which I think will address many of your desires in this area. Stay tuned.
Mylenium - I'm not asking for everything to be realtime, that would be crazy talk. What I'm asking for is once it has rendered to be able to play it back without it having to go into ram first. This is a very different thing.
Todd - Great news to hear.