I'm not currently able to replicate your problem, but I do scratch my head and wonder why you try to use AE for this at all.
The reason: there are many applications much better at compression than AE. Long-gop codecs like mp4 / h.264 suffer the most because AE is incapable of multipass compression, which can greatly improve image quality.
The standard practice is to render a lossless file from AE, and use a dedicated compression application for final delivery. Adobe Media Encoder, Sorenson Squeeze, Telestream's Episode Pro, Apple's Compressor and even Quicktime Pro will all do a better job of compression than AE.
I think AE includes various codecs just to make fast, one-off files of projects in progress.
Here's an analogy:
If you want to do precise typography, you could try using Microsoft Word, but it probably won't work very well; Illustrator is much better suited for it.
If you want to do high-quality compression, you could try using AE, but it probably won't work very well; a dedicated compression application is much better suited for it.
I used AE for encoding because I have had a bunch of old video files (some are interlaced and needed a special deinterlace operation, some needed to get edited, some files were combined into one file....) for a redesign of my web page ...
So the easiest and fastest workflow was to use AE.
You are right with the 2 pass encoding... but I usually only use 1 pass encoding because it's much faster to encode and the difference in terms of file size and quality is not that much...
I only posted this issue here for users (like me) who think that it is possible to export .h264 files for streaming purpose out of AE but as this test shows to me AE is not a good choice if you need the files for streaming.
If you don't care about the streaming.... the files from AE look as good as when encoded with Adobe Media Encoder.
My workaround was to save every AE project to disk and then I imported the AE composition directly into Adobe Media Encoder CS5 via Dynamic Link.
Sorry this is late but their is another way to solve the issue. Adobe for some reason decided to go back to the days of CS1 and render there h.264 files with the atoms at the end of the file. This means that all the metadata is at the end which is incorrect for streaming. The flash player looks for the metadata at the beginning and if it's not there it fails. There is a program called QTIndexSwapper that solves this problem: http://mac.softpedia.com/get/Developer-Tools/QTIndexSwapper.shtml
To be honest Adobe has totally screwed up h.264 compression in CS5, I've submitted several bugs and have never had a response. We have 10 copies of CS5 for our group and we still use AE CS4 because CS5 is worthless to us. We also have similar problems with AME so it's the whole sweet. And rendering 100s of videos uncompressed from CS5 and then using AME is a complete waste of time and storage space.
Thanks for the information typpeo,
I will have a look into QTIndexSwapper... seems to be a good tool.. although
I already encoded my clips with AME CS5 for streaming... and that worked...
If you do not want to render your AE compositions uncompressed only for
encoding to .h264 you also could import your AE Comps (open .aex file as it
would be a normal video file) directly into AME CS5 via dynamic link and
encode it directly with AME!
This way you safe a lot of disc space (no uncompressed file) !
But anyway it's a shame that After Effects CS5 is so limited when exporting
to .h264... I feel your pain
It pretty much doesn't matter WHAT you set in any adobe encoder/exporter. It WILL get it weird.
When leaving any adobe app go out to animation and use a decent encoder to get to your final result.
Apples compressor may be a little slower than adobes stuff but the quality is visually much better and it handles just about
everything you throw at it nicely. A GREAT alternative and it is free is MPEG StreamClip... Fantastic encoder app.
Be it AME, Premiere Pro or AE... They all do a pretty bad job at encoding most codecs.
Animation works though.
Hi, thanks, for the tip! unfortunaly I found your post late. I compressed 100 videos directly from AE .
My workaround was render in AE in H264 in a high bitrate like 6 and the compress them qith the sorenson squeeze which allows me to filter it to improve the gamma.
I also want to share/report a big BUG of After Effects for mp4 / h264 compression which drive me crazy still! I am spanish speaker so in my languaje there is something like é ó (letters with accent). If you render a video in After Effects with h264 (mp4 file) and the output modue is in a folder with a foreign character on the full path, you will feel the pain!!! the render works fine BUT ... at the end ERROR... and all the time invest lost! so, do not be foreigner!