9 Replies Latest reply on Sep 23, 2009 2:24 PM by WordRad

    blendShader with foreground object and 2 backgrounds

    WordRad Level 1

      I know with a blendShader you can pass in an additional  background image (call it background B) as an input parameter to the Pixel Bender kernel (that is, in addition to the display object and background which are passed in automatically).

       

      However I can't find any code examples anywhere.  And also my concern is how to do this efficiently.

       

      Would this additional background B have to be as large as the maximum browser size and passed in to the Pixel Bender kernel intially?  Or, would each time the foreground display object is moved, would you have to  recalculate the region of background B the display object was over. and pass that in as the new input parameter.  (Picture background B as behind background A.)

        • 1. Re: blendShader with foreground object and 2 backgrounds
          Kevin Goldsmith Level 3

          I showed how to do exactly this in the lab we did for MAX last year. See the tutorial in this PDF, it is a lot simpler than it seems: http://blogs.adobe.com/kevin.goldsmith/2008/12/materials_from_1.html

           

          Let me know if that answers all your questions...

          • 2. Re: blendShader with foreground object and 2 backgrounds
            WordRad Level 1

            Kevin:

             

            Thanks so much for your reply (and the previous one regarding the lack of

            bitwise operators in Pixel Bender).

             

            Here is what I have been trying to figure how to do in Flex for some time:

             

            (B & S) | (F &~S),

             

            where B,S,F are three bitmaps (and &,| and ~ are bitwise ops on the pixels).

             

            That's it.

             

            I've just turned over every single possible rock trying to do this,

            incredulous that given the capabilities of Flex and Flash that this was

            truly impossible.

             

            If there is ANY WAY  this can be achieved (undocumented back door -

            whatever) I would be HIGHLY appreciative.  As you may know, in something

            like Win32 it is trivial, being achievable with BitBlt.

             

            I have spent so much time trying to figure this out in Flex, that if I may,

            I'd like to review some of my experience for you.

             

            First of all, the DisplayObject BlendModes in Flex are plenty fast, and

            given that you have a BlendMode.INVERT (which is a bitwise NOT right?), why

            in the world would there not be a bitwise AND and OR as well.  As best I can

            tell that would completely solve my problem.  But anyway, for whatever

            reason, you don't have a BlendMode.AND or BlendMode.OR.  And also, I don't

            claim to know what you have to do to achieve all these things, but the

            BlendModes (such as BlendMode.INVERT) long preceded Pixel Bender's

            introduction.  [Actually if you were somehow able to tailor a specific

            solution for this, BlendMode.AND or BlendMode.OR by themselves might not be

            ideal if it required numerous copied bitmaps.]

             

            Speaking of Pixel Bender, when I finally discovered it last week (yes, I've

            sort of been stuck in Flex 2 mode for a while)  I thought, "Finally - the

            answer to my prayers."  I just could not believe it though, that bitwise

            operators were left out of it, (and as I said, I don't actually understand

            how graphics cards are a consideration here, as after all you have a

            BlendMode.Invert that predates PixelBender.)

             

            But anyway, I simulated bitwise ops in Pixel Bender with modulo operations,

            but it came out way too slow.  (I don't have a supported graphics card

            though, which brings up another question - how would you test for the

            presence of supported graphics card on a user's computer.)

             

            So Pixel Bender seems a dead end.  My most recent endeavor was to download

            the Flex source code from Adobe.Opensource, and see if it could be

            customized.  But a text search for BlendModes show that they're all  in the

            FlashPlayer itself, right?  So there is apparently no way to modify Flex

            itself to achieve what I want, right  (e.g. modified Java code or

            something).

             

            So anyway,  I'm still incredulous this can't be done ((B & S) | (F &~S),)

            and so maybe you're my last hope, here.

             

            Thanks for any help,

             

             

            Luke Braly

            • 3. Re: blendShader with foreground object and 2 backgrounds
              WordRad Level 1

              I was responding to your e-mail.  Didn't know it would end up here.

              • 4. Re: blendShader with foreground object and 2 backgrounds
                WordRad Level 1

                Someone in stackoverflow.com is telling me to do it with vectors (bitwise ops) so I'm giving that a try.

                • 5. Re: blendShader with foreground object and 2 backgrounds
                  WordRad Level 1

                  Kevin:

                   

                  As I mentioned I was looking for examples with blendShaders, which involved

                  an additional bitmap background being passed into the pixel bender kernel.

                  (i.e. in addition to the default background and foreground)

                   

                  I'm not finding that in what you provided below, fyi.

                   

                  So say this additional background B is on a display object behind both the

                  default background and foreground objects, would each time the foreground

                  object is moved, would you call BitmapData.Draw(objectB,...) to redraw the

                  portion of B the foreground was over at that moment to a bitmap which would

                  then be passed into the kernel. In general , would BitmapData.Draw be the

                  most efficient method to grab the bitmapdata of a DisplayObject at any given

                  time.  (This last question would be most relevant to me at the moment, btw).

                  • 6. Re: blendShader with foreground object and 2 backgrounds
                    WordRad Level 1

                    FYI to all-

                     

                    VECTORS is the answer.

                     

                    Probably vectors make Pixel Bender unneeded for a lot of things.

                     

                    A vector is just a superfast array, that you can seemingly traverse instantaneously regardless of its length.  (Vectors are only Flash 10 and Flex 3, so that's why I didn't know about them.)

                     

                    But many sorts of operations on bitmaps must be achievable strictly through vectors, making Pixel Bender unnecessary.

                    • 7. Re: blendShader with foreground object and 2 backgrounds
                      Kevin Goldsmith Level 3

                      If you are getting anywhere near the speed of speed of Pixel Bender using vectors, it means that you are using Pixel Bender ineffectively. If you are passing Pixel Bender data in an optimal way (datasets that are reasonably large, height > 1, your kernels written well without a lot of unecessary assignments and conditionals),  you will exceed the performance of vector significantly even on low end machines.

                       

                      If you would like to send me the code you used to call Pixel Bender, I can offer you suggestions on how to improve your performance.

                       

                      I do understand that it is hard to get used to doing development in Pixel Bender even now, but we'll be posting a lot more sample code after MAX that should help people understand how to get the performance they expect.

                      • 8. Re: blendShader with foreground object and 2 backgrounds
                        WordRad Level 1

                        Kevin:

                         

                        The operation I had to perform was (B & S) | (F & ~S)  on bitmaps B,S,F.

                         

                        The problem with pixel bender is that I couldn't do bitwise operations on pixels.  So here is what I tried instead:

                         

                            void
                            evaluatePixel()
                            {           
                                pixel4 px_s = sampleNearest(src,outCoord());
                                pixel4 px_b = sampleNearest(bgnd,outCoord());
                                pixel4 px_f = sampleNearest(fgnd,outCoord());
                                       
                                pixel4 px_sb  = px_s;
                                pixel4 px_nsf = px_s;
                                  
                        float a,b,result;

                                result = 0.0;
                                a =  px_s.r * 255.000;
                                b =  px_b.r * 255.000;
                                if ((mod(a,2.0) >= 1.0) && (mod(b,2.0) >= 1.0)) result += 1.0;
                                if ((mod(a,4.0) >= 2.0) && (mod(b,4.0) >= 2.0)) result += 2.0;
                                if ((mod(a,8.0) >= 4.0) && (mod(b,8.0) >= 4.0)) result += 4.0;
                                if ((mod(a,16.0) >= 8.0) && (mod(b,16.0) >= 8.0)) result += 8.0;
                                if ((mod(a,32.0) >= 16.0) && (mod(b,32.0) >= 16.0)) result += 16.0;
                                if ((mod(a,64.0) >= 32.0) && (mod(b,64.0) >= 32.0)) result += 32.0;
                                if ((mod(a,128.0) >= 64.0) && (mod(b,128.0) >= 64.0)) result += 64.0;
                                if ((mod(a,256.0) >= 128.0) && (mod(b,256.0) >= 128.0)) result += 128.0;
                                px_sb.r  = result/255.000;

                         

                                result = 0.0;
                                a =  px_s.g * 255.000;
                                b =  px_b.g * 255.000;
                                if ((mod(a,2.0) >= 1.0) && (mod(b,2.0) >= 1.0)) result += 1.0;
                                if ((mod(a,4.0) >= 2.0) && (mod(b,4.0) >= 2.0)) result += 2.0;
                                if ((mod(a,8.0) >= 4.0) && (mod(b,8.0) >= 4.0)) result += 4.0;
                                if ((mod(a,16.0) >= 8.0) && (mod(b,16.0) >= 8.0)) result += 8.0;
                                if ((mod(a,32.0) >= 16.0) && (mod(b,32.0) >= 16.0)) result += 16.0;
                                if ((mod(a,64.0) >= 32.0) && (mod(b,64.0) >= 32.0)) result += 32.0;
                                if ((mod(a,128.0) >= 64.0) && (mod(b,128.0) >= 64.0)) result += 64.0;
                                if ((mod(a,256.0) >= 128.0) && (mod(b,256.0) >= 128.0)) result += 128.0;
                                px_sb.g  = result/255.000;
                        .

                        .

                        .

                        //( A total of something like 175 mod calls)

                         

                        So that should explain why Pixel Bender is not the answer for me (unless there's another way to do the above).

                         

                        Here is how I did it with vectors inside of flex:

                         

                              var m:Matrix = new Matrix(1,0,0,1,-(img_x-s_r.x),-(img_y-s_r.y));
                                   
                              var bmd_fg:BitmapData = new BitmapData(img_w,img_h,false,0);  
                              bmd_fg.draw(CanvasFG,m); 
                              var vec_fg:Vector.<uint> = bmd_fg.getVector(r);

                         

                              var bmd_bg:BitmapData = new BitmapData(img_w,img_h,false,0);  
                              bmd_bg.draw(CanvasBG,m); 
                              var vec_bg:Vector.<uint> = bmd_bg.getVector(r);

                           

                              var vec_s:Vector.<uint> = bmd_img[n].getVector(r_i);
                                 
                              for (var i:int = vec_s.length - 1; i >= 0; --i)        
                                vec_s[i] = ((vec_fg[i] ^ vec_bg[i]) & vec_s[i]) ^ vec_fg[i];

                              Bitmap(img.content).bitmapData.setVector(r_i,vec_s);   

                         

                        That last operation to traverse those vectors takes something like 15 ms (for average size bitmaps).

                         

                        Note:  ((F ^ B) & S) ^ F) is slightly faster and equivalent to  (B & S) | (F & ~S).

                         

                        Also (and this is really strange)  iterating backward through the vectors is much faster than going forward.

                        • 9. Re: blendShader with foreground object and 2 backgrounds
                          WordRad Level 1

                          But it just occurs to me - there is a vector type inside Pixel Bender as well isn't there  -  but it doesn't make any difference - I still couldn't do bitwise ops properly.