0 Replies Latest reply on Jun 29, 2012 6:29 AM by GiantRobotKitten

    Please help me understand applyFilter in combination with a custom shader.


      Dear communiy. I stumbled accross some very weird behavior when using a custom shader in combination with the applyFilter method on BitmapData.


      The custom shader is (just to illustrate the problem) completely simple. It just takes the input pixels and writes them to the output. This is pretty much boilerplate.. nothing special.


      <languageVersion : 1.0;>
      kernel SimpleShader
      <   namespace : "com.example";
          vendor : "example";
          version : 1;
          input image4 src;
          output pixel4 dst;
              dst = sampleNearest(src,outCoord());


      In Flash (CS6) I'm using a simple piece of ActionScript code to create two Bitmaps and place them on the stage. I then draw a rectange to the first bitmap and use applyFilter to render that region of the first bitmap to the second bitmap!


      What's really strange now, is that this region seems to get cropped somehow and I don't have the slightest clue why this is! All my offsets/coordinates are correct and what's even more astounding is that the exact same code works fine if I replace the ShaderFilter with a Flash filter such as "DropShadowFilter".


      Here's the code (create a new FLA, put this on your first frame, set Stage size to 800x400):


      import flash.display.Bitmap;
      import flash.display.BitmapData;
      import flash.geom.Rectangle;
      import flash.display.Shader;
      import flash.utils.ByteArray;
      import flash.filters.ShaderFilter;
      import flash.filters.DropShadowFilter;
      [Embed("SimpleShader.pbj", mimeType="application/octet-stream")] 
      var SimpleShader:Class;
      var bitmap1:Bitmap = new Bitmap(new BitmapData(400, 400, true, 0x00000000));
      var bitmap2:Bitmap = new Bitmap(new BitmapData(400, 400, true, 0x00000000));
      bitmap2.x = 400;
      // this is the rectangle we want to draw!
      var rect:Rectangle = new Rectangle(120, 120, 200, 200);
      // simply fill a red rectangle in the first image
      bitmap1.bitmapData.fillRect(rect, 0xffff0000);
      var shader:Shader = new Shader(new SimpleShader() as ByteArray);
      // render the rectangle region onto bitmap2 using custom shader
      bitmap2.bitmapData.applyFilter(bitmap1.bitmapData, rect, rect.topLeft, new ShaderFilter(shader));
      // using a flash built-in shader works...
      //bitmap2.bitmapData.applyFilter(bitmap1.bitmapData, rect, rect.topLeft, new DropShadowFilter());
      // fill the outlines of "bounds" to debug that these are correct!
      bitmap2.bitmapData.fillRect(new Rectangle(rect.x, rect.y, rect.width, 1), 0xff00ff00);
      bitmap2.bitmapData.fillRect(new Rectangle(rect.x, rect.y, 1, rect.height), 0xff00ff00);
      bitmap2.bitmapData.fillRect(new Rectangle(rect.right, rect.y, 1, rect.height), 0xff00ff00);
      bitmap2.bitmapData.fillRect(new Rectangle(rect.x, rect.bottom, rect.width, 1), 0xff00ff00);


      Here's how the result looks.. the second rectangle should match the green lines.. but it doesn't! Why?


      Screen Shot 2012-06-29 at 3.28.05 PM.png