Export to MP4 (h264) causes pixelated edge

[Dunday]

Hi,

I am working on an animation in AE. In the project there is one scene where a plain blue background touches a plain red shape.

After exporting the final project to h264 via Media Encoder or AE (it doesn't make a difference which method I choose)  at this particular scene the edge where blue meets red is pixelated and jagged (see images attached). And it's really just this edge. Everything else looks fine. The preview in AE itself looks fine, too. No pixelated edge. Also increasing the bitrate doesn't affect the result. I also checked and unchecked settings like "Use maximum render quality" and "Render at Maximum Depth" with no visible change. So I tested almost every option in the h264 video settings with no success.

But exporting to a different format like Quicktime MOV with Animation codec doesn't cause those pixelated edges. So my guess is that the way the h264-compression works causes those ugly edges. Maybe it's the color contrast of red and blue? I don't know. Did anybody else notice this problem? Any ideas on how to avoid this other than using another codec? Oh and, still working with AE CC 2015.2 if that matters.

There is little motion in this scene. A slow zoom in. But as i said, in AE or with Animation codec it looks fine. So this, can't be the problem.

My basic export settings are:

1920 x 1080

square pixels

frame rate: 25

field order: progressiv

PAL

Profile: Main

Level: 4.1

Any help is appreciated.

Thank you.

Daniel

[Roei Tzoref]

After exporting the final project to h264 via Media Encoder or AE

don't use H.264 under quicktime format options in Ae. it's is buggy and low quality. use AME (which you also did)

I also checked and unchecked settings like "Use maximum render quality" and "Render at Maximum Depth" with no visible change.

these should not make a difference unless you are resizing your image in the render process and not rendering on the GPU, or require 32bit depth for further compositing (which H.264 does not support anyway - it is always 8-bit): Understanding render options in Adobe Premiere Pro - Blog - Digital Rebellion

So I tested almost every option in the h264 video settings with no success.

using match source - high bit-rate default preset should be fine.

guess is that the way the h264-compression works causes those ugly edges. Maybe it's the color contrast of red and blue? I don't know. Did anybody else notice this problem? Any ideas on how to avoid this other than using another codec? Oh and, still working with AE CC 2015.2 if that matters.

yes compression does degrade your image and contrasting pixels are a problem with mpeg compression BUT when you say that CC2015.2 is fine (you should upgrade to 2015.3 btw) this leads me to believe you are experiencing another issue which is not an encoding issue but a dynamic link bug between Ae and AME in CC2017. the bug causes AME to get the quality and resolution from the composition pop up menu in Ae. read about it here: Known issues in After Effects CC 2017

the workaround is set the resolution pop up in the composition panel to full before you add your composition to AME, or use the much better - Queue in AME feature. Try and see if that solves your problem.

[Kevin-Monahan]

Hi Dunday,

Did Roei's response help you solve your issue?

Thanks,
Kevin

[mollied93444244]

Hey Kevin,

I'm having the same problem as Dunday and the Roei's response did not fix this issue. Do you know if this has been addressed? I'm using AE CC 2018 15.0.0 (Build 180).

Here's my problem child:

The top is an Animation codec with an mov wrapper from AE. The bottom is an H.264 codec with than mp4 wrapper from AME. See how the edges are pixelated on the chair and on the red lines in the fireworks. I've tried various combinations of render settings, purged my disk/memory cache, restarted my computer, and consulted friends. At this point I'm left with going back to the illustrator and trying to tone down the red.

Do you have any ideas?

Thank you!

Mollie

[jomo5280]

I am having exact same issue - did you find a fix? thanks!

[Szalam]

The compression of an MP4 can have some issues with sharp flat colors (especially red) up against other things. It's a balancing act between design choices and the resulting...issues.

It's a limitation of the format.

Back in the day, you couldn't use fully saturated red text at all because the tube TVs it would get broadcast to would absolutely murder it. This isn't nearly as bad, but it is still annoying.

[mollied93444244]

Sorry, just seeing this now! I didn't end up finding a fix. My team decided to abandon mp4/H.264 altogether and just go with mov/Quicktime. It's a larger file size but totally worth it!

[mollied93444244]

Update: .mov did not turn out to be a solution in the long run either. The quest continues

[blackm7220802]

We are having the same issue. Our only fix is to export ProRes 422, then convert to MP4 through Quicktimes.

[derekheisler]

Funny enough I stumbled into this today. There's an issue with AME for sure. I was doing something a bit differently, as I was changing the aspect ratio and getting it to fit the frame (versus with black bars) and getting jagged edges everywhere. I ended up bringing the original video into photoshop, resized and cropped in photoshop, and rendered out using AME (inside of photoshop), and boom zero issues. This is definitely an AME bug that's never been fixed 

[Rick Gerard]

H.264 compression compresses color in blocks of 4 pixels. It also predicts the luminance values for more than half the frames. The greater the difference in the colors the more the colors are fiddled with. There are only 256 color shades per channel in H.264. You have to understand that when you design, line up horizontal and vertical edges and do the final color grading. Sometimes just adding a little noise or grain hides those funky edges. I'm actually amazed that the compression works as well as it does. It's way better than we had just a few years ago.

[quentin31]

After trying everything out, adding 2% of noise was the way. Thank You.