6 Replies Latest reply on Jul 27, 2009 2:00 PM by AIF Bob

    mod() bug?

    Christian Giordano
      On my test it seems that mod(float, float) in the case of mod(16.0, 2.0) doesn't return 0.0 so I need to compare it as < of 1.0
      Or is it more likely that also if I'm in Flash (pixelRatio 1:1) the outCoord() is never that rounded (1.0, 2.0, 3.0, ...). That's what I've been doing:

      float2 oc = outCoord();
      float4 src = sampleNearest(img, oc);
      if(mod(oc[0], 2.0)<1.0 && mod(oc[1], 2.0)<1.0){
      dst = float4(src[0], src[1], src[2], 1.0);
      }else{
      dst = float4(0.0, 0.0, 0.0, 0.0);
      }

      Cheers, chr
        • 1. Re: mod() bug?
          Christian Giordano Level 1
          I would be glad to know why this works correctly in the toolkit but not in flash (of course I tried with the mod() first):

          input image4 img;
          output float4 dst;
          //576x384
          // evaluatePixel(): The function of the filter that actually does the
          // processing of the image. This function is called once
          // for each pixel of the output image.
          void
          evaluatePixel()
          {
          float2 oc = outCoord();
          float4 src = sampleNearest(img, oc);
          float4 cleanPixel = float4(0.0, 0.0, 0.0, 0.0);

          float x = oc[0];
          float y = oc[1];

          float r = 2.0;
          float modx = x - (r * floor(x/r));
          float mody = y - (r * floor(y/r));

          dst = src;

          if(!(modx<=1.0 && mody<=1.0)){
          dst = cleanPixel;
          }else{
          dst = src;
          }
          }

          Thanks a lot, chr
          • 2. Re: mod() bug?
            Kevin Goldsmith Level 3
            thanks! we're taking a look at this...
            • 3. Re: mod() bug?
              Kevin Goldsmith Level 3
              We've verified this bug. If it is a problem in the bytecode generation, we'll fix it in the next toolkit release, otherwise, we'll get it over to the flash team to fix in the next flash release.

              thanks!
              • 4. Re: mod() bug?
                AIF Bob Level 3

                Hi Christian,

                 

                I've been looking into this, and I'm not certain that we have verified the same bug as you are reporting. I'd like to try and nail down exactly what's going on here.

                 

                First of all, the coordinates returned by outCoord are the coordinates of the centre of each pixel. This means that if your pixel size is 1x1 (which is always the case in flash, and is the case by default in the toolkit) the coordinates of the top row of pixels in your image will start like this:

                 

                (0.5,0.5) (1.5,0.5) (2.5,0.5) (3.5,0.5) etc.

                 

                Your first example where you do this:

                 

                float2 oc = outCoord();
                float4 src = sampleNearest(img, oc);

                if( mod(oc[0], 2.0)<1.0 && mod(oc[1], 2.0)<1.0 ){

                ...}

                 

                looks correct to me, and it does the correct thing in the toolkit that I am trying it on here.

                 

                You report that mod( 16.0, 2.0) doesn't return 0.0. It is possible that rounding errors mean that it won't return exactly 0.0, and it is possible that these rounding errors wil be different between the toolkit and the flash player. Floating point equality operations are always a little risky, it is more usual to do something like this:

                 

                float epsilon = 0.00001;

                if( abs( mod( 16.0, 2.0 ) ) < epsilon )

                {

                   ...

                }


                Can you post the following:

                 

                A complete kernel that shows the problem you are having

                A description of what it does in the toolkit and what you expect it to do in the toolkit.

                A description of what it does in flash and what you expect it to do in flash.

                 

                Thanks

                 

                Bob

                • 5. Re: mod() bug?
                  Christian Giordano Level 1

                  Hi Bob, the kernel I'm using at the moment is the one I posted at the start and as you say, it works correctly in the toolkit. It doesn't though in the Flash Player where all the pixels are transparent.

                  I tried also your method:

                   

                   

                   

                  <languageVersion: 1.0;>

                  kernel Sepia
                  <   namespace : "AIF";
                      vendor : "xxx";
                      version : 1;
                      description : "Grid filter"; >
                  {

                   


                      input image4 img;
                      output float4 dst;

                   

                      void
                      evaluatePixel()
                      {
                          float2 oc = outCoord();
                          float4 src = sampleNearest(img, oc);
                          float4 cleanPixel = float4(0.0, 0.0, 0.0, 0.0);
                         
                          float x = oc[0] - 0.5;
                          float y = oc[1] - 0.5;
                         
                          float r = 2.0;

                          float modx = abs(mod(x,r));
                          float mody = abs(mod(y,r));
                         
                          dst = src;
                         
                          float epsilon = 0.00001;

                   

                          if(!(modx<epsilon && mody<epsilon)){
                              dst = cleanPixel;
                          }else{
                              dst = src;
                          }
                      }
                  }

                   

                   

                  But I get the same result. Just test it in Flash.

                   

                  Cheers, chr

                  • 6. Re: mod() bug?
                    AIF Bob Level 3

                    Hi Christian,

                     

                    I am sorry that this has taken so long to sort out, but I think I have some more information.

                     

                    I think that there is a bug with the boolean not operator. You have these lines:

                     

                            if(!(modx<epsilon && mody<epsilon)){
                                dst = cleanPixel;
                            }else{
                                dst = src;
                            }

                    Removing the not and reversing the if and else clasues gives you this:

                     

                            if( ( modx < epsilon && mody < epsilon ) ){
                                dst = src;
                            }else{
                                dst = cleanPixel;
                            }

                    which I think works better.

                     

                    I also think you might still be running into rounding errors with the mod, but with the values before you do the mod, not afterwards. If you execute this code:

                     

                    float x = 2.5;

                    float x1 = x - 0.5;

                    float modx = mod( x1, 2.0 );

                     

                    You would expect x1 to be equal to 2.0 and modx to be equal to zero (or within epsilon of zero).

                     

                    If, however, as a result of a slight rounding error x1 were actually equal to 1.9999999999, modx would then end up as 1.99999999 - not within epsilon of zero.

                     

                    I think you'll need to take extra care in looking at the results of a mod operation.