5 Replies Latest reply on Nov 12, 2011 3:48 AM by kaczorefx

    white border in composite


      I have a series of PNG images (character and background) rendered out of Autodesk Maya (with Mental Ray) for compositing in AE.

      They are the DiffuseNoShadow, Indirect, and Shadow pass. Premultiply was disabled.


      I imported the sequences into AE, initially 'Straight'.  For a foreground character, I screen the Indirect pass over the Diffuse in it's own comp, but notice a dark border.  Interpreting the Indirect layer as being 'Premultiplied' fixes things somewhat, but leaves traces of a faint white border.


      I would add a 'Matte Choke' to remove it, but since the layers have a "holdout" (from mental ray) doing so would create a black where the pixels were choked.  Also there are some narrow areas I can't choke as they'll be removed.


      Anyone have a solution to my border issue? My appreciation if you do!

        • 1. Re: white border in composite
          Rick Gerard Adobe Community Professional & MVP

          In most cases you should be using straight alphas. That should take care of any colors generated when you pre-multiply the alpha. The small white borders are common when interpreting straight alphas as pre-multiplied. The dark edges that appear with you screen the indirect pass over the diffuse mean that the indirect pass is a little smaller than the diffuse pass. A screenshot or maybe a frame from each of the render passes would help me figure out exactly what's wrong. My first idea would be to add some light wrap to the diffuse layer.

          1 person found this helpful
          • 2. Re: white border in composite
            john.slately Level 1

            hmm...one shouldn't be smaller than the other, so I wonder why.


            your lightwrap suggestion, with the addition of edge blur and premultiplying the indirect layer minimized the issue. Thank you.

            • 3. Re: white border in composite
              Rick Gerard Adobe Community Professional & MVP

              It's smaller because of antialiasing settings. If both passes were set to none then the layers would be the same size, but because you probably have curved or angled edges each pass renders the differing pixel values on the edges with slightly differing accuracy. This is because of the different pixel values in each pass. That's one reason you should always render out straight alpha's. It's also the reason that you get the darker edges when you interpret straight alphas as premultiplied.

              • 4. Re: white border in composite

                I'm using passes in Maya, so the indirect and diffuse both share the same settings.  Which has 'premultiply' disabled.

                but I undersand how antialiasing could be a culprit.  I'll look into it some more and get back to this thread. Thank you.

                • 5. Re: white border in composite
                  kaczorefx Level 1


                  I actually wrote a plugin for AE for dealing witht this very issue




                  Watch the full tutorial, to see how it works - I forgot to show it in the demo video ;(