6 Replies Latest reply on Aug 23, 2010 5:56 PM by FuzzyMonkeyMan

    Raster export options..




      I am a programmer working with artists using After Effects (+Mocha) and attempting to get clean rotoscope masks in some type of image format.


      The rasterized image are color-coded so each shape has a unique color, this is mostly working as expected.


      I attached some extreme closeup shots of the image pixels after export to illustrate the problems, I believe there may be a simple solution to this using the right export/raster options..



      The Minor Problem:


      The problem is that the pixels between shapes are blended together, so instead of (example) 5 unique colors for 5 shapes in the image, the output image might have hundreds of "in-between" colors where the pixels are blended. This is a natural anti-aliasing that I would like to disable, but the artists do not know of any way to do this in AE..


      It's not a deal-breaker, this can be worked around to some extent by  (a) identifying the dominant/legitimate unique colors and filtering out  the rest to the nearest match.







      A Bigger Problem:


      The real problem comes in when pixels that are well within the borders between shapes have minor differences, such as 2 adjacent pixels that are off by a single RGB value.


      I made an image for the artists to illustrate the problem:




      These can also be filtered out, but that tends to destroy legitimate detail. There is no single automated solution that resolves every pixel perfectly without breaking others (especially when the selected "unique" colors are very close)..



      The Goal:


      I really just need a per-pixel mapping of which pixel goes with which outline. I assumed that the color-coded layermasks would be the most portable solution for this type of information.



      The Solution:


      The artists I am working with state that After Effects is unable to export clean mask images. If this is true, my other options are to (a) pick some alternate format for shapes, load the shape data and rasterize it myself, or (b) get up to speed with the AE SDK and try to write an export filter.


      Before I go to those lengths I thought I should sign up in these forums and ask if this true (that AE is unable to render out clean shapes), and if so, what recommendations others might have.


      I'm open to any solution that it takes (failure is not an option), if anyone has any suggestions or information on how to approach this with AE I am glad for any help..


      Thanks in advance


        • 1. Re: Raster export options..
          TimeRemapper Level 4

          There are a number of ways to try and define "aliased" mattes. One way that may be easiest for you depending on the nature of the project is to set the layer/render quality to "draft".

          • 2. Re: Raster export options..
            Rick Gerard Adobe Community Professional & MVP

            In a practical sense the only way that a curved edge made of pixels can appear to be smooth is to:

            1. Have the pixels so small that your eye can't discern the jagged edges (aliasing) or,
            2. Blur the edge so that your eye is fooled into thinking the edge is smooth

            Computer monitors have a very short dot (pixel) pitch so that the type appears to be smooth. Use a CRT from 1990 and see how pleasing the type is to look at. But even todays very high resolution displays have built in antialiasing that smooths out the edge of the type.


            Your question seems to be "Can AE export clean masks?" It sure can. The higher the bit depth the cleaner, but a clean matte does't mean no grey in the alpha channel. It means an edge that appears smooth. Even with the cleanest edges I'm always applying a form of Light Wrap to the image to smooth out and blend a layer into the background so that the composite is believable. Light does this. I'm looking outside at the sun filtering through my Laurel hedge and the edges of the leaves have a little of the blue from the sky around their edges. To make a believable composite you have to blur the edges and wrap pixels from the background on top of the foreground.


            If you really want to clip a matte to an exact pixel value you can set that up easily with levels. The attached image from the comp window is of two identical circular masks on a white solid positioned at uneven pixel values (off by an whole pixel value by .5 in one direction and .8 in the other). I simply applied levels to the layer on the right and set the offset black to .5000000 and white to .500000001 to clip the edge of the mask to a very precise pixel value. Which one looks like a circle? Which is the cleaner mask? The one on the right is clipped to very small pixel value. Change the number of decimal places to 4 places and the matte will be different, so you might say that the mask on the right is more accurate than the one on the left. The problem is that the one on the right would not composite into a scene easily, so IMHO the one on the left, the one with the grey values in the edges, the one with the interpolated pixel values, is the cleaner matte.


            I hope this helps.


            Screen shot 2010-08-22 at 4.16.06 PM.png

            • 3. Re: Raster export options..
              FuzzyMonkeyMan Level 1

              Hi Rick-


              I believe your example is exactly what I mean by "clean", the pixels go to one or the other vs. the interpolation between them, it's not visually smooth but there is a clear separation of values..


              I can't verify the solution until I get a test case back but you're describing the issue perfectly...I don't even have AE so it's difficult for me to describe any specific steps to remote users, this should be enough info to straighten things out..


              I'll probably flag this as the solution after I can verify the results, I was pretty sure that AE could do this with the right settings, it sounds like you know what I'm looking at..

              • 4. Re: Raster export options..
                TimeRemapper Level 4

                In addition to Rick's magnificent (as usual) explanation of the underpinnings of not only reality, but the digital replication of it, here are two examples of what the difference between a "draft" setting, and a "best" setting will yield. Both are circular masks drawn on a white solid.





                and, "best":


                1 person found this helpful
                • 5. Re: Raster export options..
                  bogiesan-gyyClL Level 3

                  > The artists I am working with state that After Effects is unable to export clean mask images. If this is true, my other options are to (a) pick some alternate format for shapes, load the shape data and rasterize it myself, or (b) get up to speed with the AE SDK and try to write an export filter.<


                  Welcome to the family.

                  I'm having a bit of trouble with this claim but not because of my competence or experience, simply from our anecdotal history around here.

                  Major houses use After Effects at 2k and 4k resolution and even IMAX to create totally invisible masking and effects.



                  • 6. Re: Raster export options..
                    FuzzyMonkeyMan Level 1

                    Hi bogiesan, I have to admit I'm being a little facetious in my post, I was absolutely positive this could be done in AE but I don't have a copy (or access to s system with one) and I don't know how else to explain it to the guys I'm working with remotely..


                    I asked the artists to disable anti-aliasing and they said it "could not be done", so I figured I would take them literally at their word and bring it up on the AE forums, I'm glad I did..


                    I'm still waiting on a test case to verify so I can mark this as solved, the test renders from TimeRemapper and Rick Gerard demonstrate exactly what I was looking for so I expect it will work for what I need..