Either the good old H.264 Gamma problem or simple quantization errors which you might probably avoid by working in 16bpc or 32bpc. Also keep in mind that AE can use color management and you may already see those differences when setting the comp viewer proofing to you monitor profile. Generally, though, such color issues are extremely difficult to track down. Things like your graphics hardware's acceleration features, VLCs selected decode method, color profiles and other color tweaks can influence this, both in mere perception of colors as well as actually skewing them. Start by working in higher bit-depths when re-importing files into AE and tippi-toe your way to the aforementioned color management stuff and so on...
...and to add to Mylenium's comments, H.264 is a DELIVERY codec. It isn't a lossless intermedate, aka production, codec. H.264 is lossy, and bad things will happen with repeated recompression.
The chances are good that you're not going straight to final delivery from AE: there's almost always another application that will utilize your AE work. You should either render out of AE (using the Render Queue) in a lossless codec if there's any kind of codec question -- I like QT's in the PNG codec. It's also okay to render in the in thebest-acceptable codec for use in the next application. If that happens to be H.264, someone could be sandbagging their delivery specs to save storage or it isn't much of an application.