The only objects that you can move the camera in on and not get pixelization are vector objects like Illustrator file, Shape layers, or masked solids. In all other cases, when the camera is the same distance away from the layer as the zoom value (object at comp center and z =0, camera zoom value 1000, camera centered in the comp and z = -1000) the layer is at 100% scale and you're seeing pixel for pixel. If you move the camera closer by half the scale effective scale becomes 200%. It doesn't take much to end up with a mess.
Most of the time, if I'm going that close, I hide the big chunks with motion blur.
As far as speeding up working with large layers I'd look at either scaling down the resolution of your comp window (set to 1/4) or learn about using proxies. Get the motion the way you want to to look, turn on motion blur and render a test.
If you've got really huge files you can do the crop and scale trick and then stack them in the same 3D space and cut between them. Here's the trick. Start with an image that's 10X the size of your comp, duplicate it and scale it down 50%, duplicate that and scale it down 50%. Now crop the original image to 25% of the original size, crop the second to 50% of the original size. You can do this with canvas size. Leave the last one full size.
This gives you three images that you can scale up in AE but position far away from the camera. The last one becomes the wide shot, and as you move in you cut to the next closer, then the next closer crop. This keeps the size of the images about 2X the comp size, saves a bunch of render time, and makes your composition much easier to handle. The trick is to use the scale property on the first two copies (full wide and middle zoom) so that the image size matches the next layer below it. The widest image is scaled 400% (still less than pixel for pixel because of it's distance from the camera), the second scaled 200%, and the third left at 100%. You won't have pixelization issues, you will have a more managable composition, and your render times will be reduced.
I hope these suggestions help.
Thank you Rick, these are all very helpful ideas. I'll give it a shot with both scaling and proxies.
I actually am surprised of how incapable computers still are to face 3d compositing...