Aliasing and Jagged Edges with Illustrator Files - Please Help!

[flowsnone]

Hey Folks,

I'm working on some feature film graphics animations where I have to put Illustrator Shapes into perspective. I have some real bad Aliasing problems.

I compared this by rendering the graphics flat and comping them with nuke, which gave me much better results. I already activated the "better" AA Option in "interpret footage", activated the sun, used the curve thingy in the layer manager. I don't know what to do. Is AEs Anti Aliasing really that bad? In Nuke, also the type was much much clearer when I put it into perspective. Everything looked better overall. But I dont want to switch to Nuke for comping as I have a lot of animation going on and this would not be a good workflow for me. What Can I do?

[flowsnone]

Also worth mentioning: I already tried blend mode "alpha add" and my illustrator files are HUGE

[Mike_Abbott]

You are viewing this at 200% of course...

You have fx applied to those layers. How does it look with fx switched off?

As a work around: could you get away with a tiny amount of fast box blur to smooth the edges?

[flowsnone]

Thanks for your response Mike!

Yes I know its 200% so you can see the pattern. In Nuke its a lot better anti aliased.

I have no FX on the layers exept some position wiggle scripts, which have controllers.

I tried blurs but thats not what I want.

[davidarbor]

Back in Illustrator have you selected your art and made it "Pixel Perfect?" Is it possible that the lines are on a half pixel? Also, what happens if you right-click on the Ai layer inside of Ae and convert it to shape layer and then check the position? If it's not on a whole pixel that could be a problem, although, when that is an issue you usually see a ghosting effect, not so much a sharp edge with bad anti-aliasing.

[flowsnone]

thanks but I have indeed used illustrators pixel preview aswell as snap to pixel! just like u should. everything is perfectly fine.

but a layer rotated in 3d space will never be pixel perfect.

I guess its just After Effects screwed up or not even existing anti aliasing.........this really sucks

[davidarbor]

Ah, didn't realize it was rotated in 3D space. Did you try converting to a native Ae shape layer just as a test?

[flowsnone]

yes I tried that. I also created a AE shape layer which behaves the same way. I guess this proves that it isnt an illustrator content creation error.

[Kevin-Monahan]

Flowsnone,

How did you end up solving this issue? I'm curious to find out. Please let us know.

Thanks,
Kevin

[flowsnone]

Went to nuke

[brandspanking]

Had a laugh, thanks!

[angie_taylor]

Make sure to click on the Collapse Trasformations switch and change the Anti Aliasing Settings to Best

[Roland Kahlenberg]

Works at 200% and complains of anti-aliasing issues — pure genius at work. Dude! We all work in 3D at some time and we don't have any issues when rotating Shape Layers or Illustrator files.

[flowsnone]

200 just to make it more visible to you guys. I compared at 100%. Nukes AA is WAY better than AE. Just try it yourself. You can use any shape or file with straight lines. Rotate it in space and watch what happens. Do the same thing in Nuke, barely visible aliasing.

[flowsnone]

I did both, doesnt help at all as my shape is already over 2000px wide in a 2k comp.

[Roland Kahlenberg]

It does not matter how large your image is. What's important is if you've Continuously Rasterized the AI file if you have scaled it up.
I also suggest that you go to Preferences>Previews and ensure that the settings there are set to More Accurate. Then, ensure that you do not have Pixel Aspect Ratio correction enabled.

AND, please post a side-by-side comparison of what you get from Nuke and AE.

[flowsnone]

Its scaled down from 2k so rasterize wont do anything for me. The more accurate option is turned on. Everything was mentioned before above

[Roei Tzoref]

guess we'll have to see for ourselves how nuke handles it. as RK asked - show it to us.

I would check your artboard options, see that its reference point is in whole pixels. if not, set it to whole pixels and save.

other than that, in my opinion the antialiasing you are showing doesn't look anything out of the ordinary. take the same shape in illustrator and turn on pixel preview and magnify to 200% and you will see the same antialiasing steps. the only thing I can recommend for better anti-aliasing is actually don't use continuously rasterize but instead convert to shape which if fact gives smoother antialiasing than CR of Ai layers, and also check the layer's interpret footage for more options -> more accurate on anti-aliasing. I see you set the quality of the layer to bicubic, see if bilinear looks better.

other than than you can do what most of the users do if this situation get's annoying and that's to soft the edges with some box blur with lower iteration and lower blur radios and see where that get's you.

[Roland Kahlenberg]

Roei, posting a 200%, exaggerated  image, to complain about exaggerated aliasian seems to be his idea of telling the rest of us that we're wrong or lesser to his ideals.

😄

[Roei Tzoref]

I dunno Roland, maybe Nuke really does it better we need to see and maybe we should aspire to cleaner lines! antialiasing is annoying and rotated art on some degree introduce ugly steps that we should all try to avoid our best. this is more apparent when the graphic is supposed to stand still. it is true for example that continousely rasterize is worse than using a shape layer because there are less antilaiased steps.  here's an example sent by Robert Kjettrup, a colleague of us both:

and also interpret footage can make cleaner antialiasing on some cases. so if all these are checked, I don't see any difference from any of the other Adobe's software's implementation of antialiasing. Maybe Nuke did something else? I am guessing some sort of softening we can also do ourselves using a bit of blur with the right amount. 

[flowsnone]

I have prepared a comparison setup and will upload a comparison file on monday. You have like 10 different filters in nuke to deal with pixel interpolation but even with the standard one it looks better. Resterize, like you said, sometimes makes things even worse. In my case its better to work with a huge .AI with rasterize off.

[Roei Tzoref]

in Ae we only have 2 ways of sampling pixels: bilinear and bicubic. maybe we need the other seven?

you choose bicubic

which is usually best for photographs and not graphics, and is more resource intensive. but I don't expect it to make a difference in this case, you should try to see if it does nevertheless.

thanks for the update, looking forward to see what we will conclude in the end of this

[Rick Gerard]

Video is limited to the pixel grid, there are no vectors in the final product. If a line does not perfectly line up with the grid it will appear to have stair steps. Take a black line one pixel wide on a white background that starts precisely on 10, 10 and ends precisely on 266, 11. In 256 pixels the line moves up 1 pixel. Try this with a shape layer.  The line will be interpreted and except for the exact middle and each end, the line will be 2 pixels thick. The color of each of the two rows will be divided into sections based on the color depth and available shades as top row get progressively lighter and the bottom row gets progressively darker. If you have a 10-bit monitor you can see the stair-stepping in the colors change as you increase bit depth in the project, but if you have an 8-bit monitor you are limited to 256 shades. You will never see the improvement in antialiasing on an 8-bit monitor. Changing the interpretation rules for aliasing only changes the rate at which the colors change.

So what's the point of that paragraph? The point is that there will always be stair-stepping (aliasing) and the quality of the antialiasing can only be increased by increasing the bit depth but you can't see any increase in quality unless you have a high bit depth monitor.

So what's the point? The point is that if you want perfect lines in a video the lines must be horizontal and vertical and perfectly aligned with the pixel grid. Otherwise, there is no such thing as a perfect line. If you want to increase the quality of the antialiasing, increase the bit depth, but you will not be able to see it unless you have a high bit depth monitor and when the video is compressed to an 8-bit codec for delivery the stair stepping is going to return. There is no way around this.

In practical terms, the only way to judge the quality of the lines in your video is to watch the video at 100% scale in real time. Obsessing about the antialiasing in a still frame viewed at 200% or 400% is a complete waste of your time. If you don't like the edges you are getting either make them move or change the design. If a line appears to change widths as it moves or it moves erratically then change the speed because smooth movement of fine detail will only happen if the movement of the line is a whole number of pixels per frame and the frame rate and speed of the motion does not create a stroboscopic effect on the line causing it to judder. If you are stuck with a certain speed that causes juddering your only option is to try and hide the problem with motion blur.

Sorry, but those are the facts. I have avoided this thread because has been a discussion about a limitation that cannot be overcome when working in video. You are stuck with the pixel grid and the frame rate and you have to work within those limitations.

[flowsnone]

Thanks for the insight, this is very interesting to me! I understand most of what you are saying and that makes complete sense. However, when you switch filters in nuke, there are some that look better than others. I understand that there will be always stepping but the distribution can vary and for some angles, some filters are better than others. I will show them all on monday! But again thanks for the explaination. Have a nice weekend!

[Kevin-Monahan]

Hi Flowsnone,

Let us know what you find out when you return to your project.

Thanks,
Kevin