quicktime:animation codec - it's lossless, so there's no visible compression, and smaller file size (depends on your footage).
Rowby: A format is not always a definition of quality. You can think of formats (at least, AVI and Quicktime formats) as containers or boxes. The quality is defined by the codec used to encode the content inside of them (in other cases, a format does imply a specific codec, like M2V or M2T files which are always used to store MPEG-2 compressed video).
AVI uncompressed is super high quality, but perhaps a rather inefficient way of storing information. It has a very, very high data rate, even for uncompressed video.
AVI 8-bit YUV 422 is also uncompressed but has a somewhat lower data rate (still, about 20 Megabytes per second for Standard Definition, don't even mention HD!) and nearly the same quality.
Quicktime - Animation also has uncompressed quality, but having lossless encoding the data rate can vary from very low to extremely high depending on the nature of the source. For a green screen shot, I'd expect it to be about the same size as AVI 8-bit YUV 422.
So, it's up to you.
No matter what format/codec you choose, an uncompressed codec it's not going to improve the quality of the original file from the camera. So, one thing you could do is - avoid exporting a file at all. Just copy the clip in the Premiere Pro timeline and paste it in the After Effects Composition. It'll keep the same the in/out points you set in Premiere.
The best starts with the camera and recording system. Any system that gives you 4:2:2 color at 10 bit as original at a high data rate is best. Well almost best. Best is 4:4:4 color at 32bit, but you must have a pile of cash to capture this way.
The worst modern acquisition formats are the mpeg systems used by phones and some digital still cameras. For cameras that independent film makers can afford, let's say under $10K, I'd say that the best, meaning highest color depth and least compression goes something like this. There may be some room for debate on the order of the Panasonic and Sony P2/XDCAM but the new AVC-Intra format is, for now, at the top:
- Panasonic AVC-Intra (the replacement for P2)
- Sony XDCAM and HDCAM SR (new generation)
- Panasonic P2 DVCPRO
- DVC Pro 100 SD
- DVC Pro 50 SD
- AVCHD (though this is a HD format I've put it below sd formats because of compression)
I've left off analogue formats because they require specialized capture cards. I also left off the Red and the Canon and Nikon still cameras that capture HD because the Red is too expensive for the list and the still cameras use a compression scheme that I haven't played with yet. I think that the Canon 5D Mark II may produce footage that will key quite well but don't know where to put it in the list.
Once you have the original footage you can either work with the native formats or transcode to your favorite lossless format. I prefer Animation Codec Quicktime because it is universally available to and simple to transfer between platforms. Also good are sequential tiff or even png formats. Many folks use jpg compressed Quicktimes. I don't know many pros that use avi's because uncompressed are so huge and the compressed avi's are pretty messy.
For the production pipeline to keep working you almost always have to stray from the native codecs because they do not support alpha channels. If you choose to work with P2 (MXF) files, which AE CS4 handles just fine, then you will have to render to a codec that supports alpha channels or render separate alpha channels to bring your footage into a NLE or Flash and with transparency preserved.
I hope that answers your question. You can't easily fix a poor quality original by simply transcoding to a lossless format.
While the others already have provided valuable info, the decicive factor is how you actualyl captured your stuff in Premiere. If it's just any of Premiere's native formats (HDV, DV, XDCAM, AVCHD, MPEG-2 etc.), you should be able to import them 1:1 without any extra conversions into AE via MediaCore's shared routines. This would save time and drive space and since it neitehr adds nor removes any data, it won't impact all your keying work. The only downside may be, that since AE always needs to decompress multiple frames to reconstruct a given frame, work may be slightly slower. Also some flavors of these formats are not compatible with AE, so then a conversion becomes inevitable. If you must convert such stuff, use losslessly compressed formats such as Quicktime Animation or PNG. If you have captured your video using specialised CoDecs such as the ones from Blackmagic, AJA or Cineform, you normally can leave them as is. They will already be embedded as AVI or MOV container files and thus be usable by any programs that understands Quicktime or the Windows Media Architecture, respectively. This would also be even more advisable, if you capture to those CoDercs with 10/12/14bit. Any conversion to 8bit CoDecs would apparently already reduce the color range and therefore might influence keying quality.