2 Replies Latest reply on May 12, 2012 4:46 AM by John Blackburne

    Odd PB alpha handling in Flash

    John Blackburne

      I encountered an odd problem with a shader I wrote this week. I eventually achieved the effect another way but came back to it today to try and work out what was happening, and it's even odder than I thought.

       

      My test shader is as follows

       

              float2 relativePos = outCoord();

              pixel4 inCol = sampleNearest(oImage, relativePos);

              dst = pixel4(0.5, relativePos.x * 0.005, relativePos.y * 0.005, inCol.w);

       

      All it does is generates a colour and combines it with the alpha of the source image. The problem is this seems to produce odd results for intermediate alpha values: they seem often to get greyer. So I wrote an app to test it. The app is here

       

      http://wonderfl.net/c/dLHH

       

      And all it does is generate a bitmap with a lot of alpha and no colour (at left) then uses applyFilter to generate another bitmap (at right). Although the pattern is roughly preserved the intermediate alpha values look very odd. Looking closely there are distinct grey borders around the holes e.g. at the right.

       

      alphaTest.png

      This is a simple test case to highlight the problem but it's apparent and obvious in much less extreme images. It's noticable in the toolkit too, althogh harder to spot against its background.

        • 1. Re: Odd PB alpha handling in Flash
          Kevin Goldsmith Level 3

          Images in Flash have their alphas pre-multiplied before being handed to Pixel Bender for processing, this would account for what you are seeing, I think.

          • 2. Re: Odd PB alpha handling in Flash
            John Blackburne Level 1

            It does, sort of. I think I know what's going on now.

             

            I don't care about the input side of it: I'm only using the alpha of the input image, not the colour, so any premultiplication is being disregarded. I can throw away the input alpha entirely and it still happens with a shader-generated alpha. Whatever it is is happening on output.

             

            It seems it's also scaling the output, attempting to reverse the pre-multiplication so a pass-thru filter

             

            dst = sampleNearest(oImage, relativePos);

             

            would have no effect. To undo this I need to do the following

             

                    float2 relativePos = outCoord();

                    pixel4 inCol = sampleNearest(oImage, relativePos);

                    float fA = inCol.a;

                    dst = pixel4(0.0, relativePos.x * 0.005 * fA, relativePos.y * 0.005 * fA, fA);

             

            i.e. multiply each channel by the alpha which seems to work.

             

            I can't see this documented anywhere but the toolkit provides a clue: the Preferences have a 'Premultiply/unmultiply color by alpha' which says "Flash Player premultiplies each color by the alpha channel. With this option checked, you will get a better approximation of running in Flash Player." Except it seems to do nothing and be always on. After that a bit of experimentation suggested what unmultiplying is.