4 Replies Latest reply on Sep 1, 2011 10:26 AM by AIF Bob

    Need Help Configuring ShaderJob

    tcorbet Level 1

      01.  I am attempting to blend two ImageSnapshots.

      02.  If I write those out as .PNG files, I am able to get the blend that I desire from the custom PBK that I have written and tested with the 2.5 Toolkit.

      03.  The snapshots are produced in an AIR application and the desired processing is to perform the blend in a background ShaderJob.

      04.  I have no trouble creating the Shader from the embedded PBJ, but I am quite confused as to how to configure the ShaderJob.  The first error I received was #2166.  That caused me to re-read the instructions concerning ShaderInput which, admittedly, I had not been able to understand:

       

      If the shader is being executed using a ShaderJob instance to process a        ByteArray containing a linear array of data, set the ShaderInput instance's        height to 1 and width to the number of 32-bit floating        point values in the ByteArray. In that case, the input in the shader must be defined with        the image1 data type.

       

      05.  Since my kernel works with input sources typed as image4 [at least when being used against the file versions of the bitmaps], I did not want to change that kernel, rather I thought the diagnostic error message might indicate that I had failed to provide width and height values for the two ShaderInputs.  That resulted in error #2165, and that is why I am posting.  Clearly I need to understand how to create the job to run against the in-memory image data as differently from how it is run in the Toolkit IDE.

       

      A.  Is the phrase "a ByteArray containing a linear array of data" meant to indicate that there are some ByteArrays which do contain a linear array of data and other ByteArrays which do not contain a linear array of data, or does the phrase mean that any use of a ByteArray results in a linear array of data as seen by the kernel's input handler?  If it is the former, can you provide some guidance on knowing when a ByteArray does or does not contain a linear array of data.  More specifically, for my use case, is the ByteArray which is available as a property in an ImageSnapshot linear or non-linear?

       

      B.  Assuming that my ByteArray must be seen by the kernel's input handler as a linear string of bytes, so that I must re-write my kernel to handle image1 rather than image4 input, does that require any changes to the use of the intrinsic outCoord function or sampleNearest?  [Searching this forum for ShaderJob, indicates that there was a bug concerning image1 versus image4 in an earlier version.  Is that fixed, or are there still 'disconnects' that must be mastered in order to successfully handle bytearrays as shader jobs?]  Or, as it seems, does sending in a linear stream only change the 'behind the scenes' parallelization strategy employed by the FlashPlayer when running the ShaderJob?  Since I am planning to run the job asynchronously, I am not too concerned about elapsed time, but are there other configuration options to be considered that might result in greater throughput on the multi-core hosts where my application will usually run?

       

      C.  Alternately, if a ByteArray may, under some circumstances, be seen by the kernel's input handler as non-linear, what settings of input.width and input.height should be set?  Again, to go from the general to the specific, if there is any guidance on specifying the dimensions of a ByteArray created by ImageSnapshot, I would appreciate that advice.

        • 1. Re: Need Help Configuring ShaderJob
          tcorbet Level 1

          If having source image content in the form of a ByteArray means that it must be fed to PixelBender as a 1-dimensional, linear stream of bytes, it seems that I would have to determine the meaning of a scalar pixel1 by  performing modulo 4 arithmetic on the x value returned by the outCoord() function.  Then, perfoming some address arithmetic, I could 'look-forward' and 'look-back' to effectively re-construct a pixel4 value in order to perform some conditional logic based on the value of the rgba values.

           

          All-in-all, it's not a very attractive proposition.  So, while I still don't know nearly as much as I would like to know, I have decided that the only practical alternative is to provide PixelBender with pixel4 in the form of BitmapData.  Fortunately, ImageSnapshot provides captureBitmapData() in addition to the captureImage() method that I was using to produce .png file output.

           

          Following that approach, I am able to get blended output, so, as regards this posting, I guess I should close out my request for help.

          • 2. Re: Need Help Configuring ShaderJob
            AIF Bob Level 3

            I'm not entirely sure I understand the question - I'll do my best to answer it but please correct any misunderstandings I have. From what you've described I think you should be using a BitmapData object since you have 2D images. There's an example on this page:

             

            http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/display/ShaderInp ut.html

             

            That shows the input data being set like this:

             

            myShader.data.src.image = new BitmapData(50, 50, true, 0xFF990000);

             

            The advantage of using a BitmapData object is that all of the work to handle the offsets within the image and the number of channels is done for you automatically. The only reason to use a ByteArray would be if the data was not a bitmap, or was not in that form. As you point out, if the data in a ByteArray actually represents a 2D image and you want to treat it as a 2D image, you then have to do all of the magic to convert coordinates into the right form by hand.

             

            Bob

            • 3. Re: Need Help Configuring ShaderJob
              tcorbet Level 1

              01.  Thank you for your helpful reply.

               

              02.  I eventually, mostly, came to understand what the technical writer had meant.  I also know that Adobe is a pretty big place with lots of different people working hard to provide product solutions, so I know it is not your job to work on what I recommend as the fix.  But since the use of the Commenting facility on the Actionscript Reference pages all too often only results in chit-chat that is not so helpful, may I ask that you forward this to the person(s) who own the flash.display.Shader classes.

               

              Instead of:

               

              "If the shader is being executed using a ShaderJob instance to process a        ByteArray containing a linear array of data, set the ShaderInput instance's        height to 1 and width to the number of 32-bit floating        point values in the ByteArray. In that case, the input in the shader must be defined with        the image1 data type."

               

              I suggest:

               

              "As a special use case, a ByteArray may be passed as an input to a Shader in lieu of the BitmapData use case shown above.  The resulting linear array of data must be given a height of 1 and a width equal to the number of 32-bit floating point values in the ByteArray, i.e. its length divided by four.  Correspondingly, the type of the input in the kernel code must be defined as image1 in lieu of image4."

               

              I would then add a reference and a link to material describing the use of pixelbender for 'number crunching' applications.

              • 4. Re: Need Help Configuring ShaderJob
                AIF Bob Level 3

                I've passed this on to the correct team and they've added it to their backlog.

                 

                Thanks

                 

                Bob