15 Replies Latest reply on Mar 4, 2013 4:13 AM by Trevorׅ

    Funny modulus result? If this a rounding error?

    TᴀW Adobe Community Professional & MVP

      Hi,

       

      Strangely, I'm getting the following result in the ESTK Javascript console:

       

      492.1 % 13.3

      Result: 13.3

       

      Clearly the result should be 0 (since 492.1 is a multiple of 13.3).

       

      What's going on? Is this because of floating-point arithmetic? In any

      case, what can I do about it to get an accurate result?

       

      Thanks,

      Ariel

       

        • 1. Re: Funny modulus result? If this a rounding error?
          MrTIFF Level 3

          I suppose that technically it's an ExtendScript bug ... probably the '%' operator is deciding that 492.1 / 13.3 is 36 with a remainder of 13.3, instead of 37 with a remainder of 0.

           

          But of course usually '%' is used with integers ... I wouldn't trust it with doing floating point arithmetic correctly anyway. In this particular case you could test for a remainder of either 0 or the divisor, but I'm not sure that I would trust that, either. I think that I'd need to know more about what you're trying to do before coming up with a reasonable suggestion.

           

          Or of course you could do your own modulus operator, but I'm not sure I would trust even that:

           

          function mod (a,b)

          {

               var m = ((a/b)-a)*b;

          }

          • 2. Re: Funny modulus result? If this a rounding error?
            MrTIFF Level 3

            Sorry, hit 'Send' unintentionally ... I think that mod(a,b) should be something more like:

             

            ((a/b)-Math.floor(a/b)) * b

             

            I'm sure there are more elegant implementations ...

            • 3. Re: Funny modulus result? If this a rounding error?
              TᴀW Adobe Community Professional & MVP

              Thanks Stephen for your input.

               

              I don't think it says anywhere that the % operator should only be used

              with integers? At any rate, it seems to work fine with decimals in most

              cases.

               

              Your mod function seems rather strange to me: try a = 9, b = 3:

               

              a/b = 3

              3-9 = -6

              -6 * 3 = -18

               

              which clearly isn't the right answer for 9 % 3 ?

               

              At any rate, I've meanwhile established that this is a floating-point

              arithmetic error. Because, once again in the ESTK console, the following

              shows that what looks like 13.3 isn't:

               

              492.1 % 13.3

              Result: 13.3

               

              but....

               

              (492.1%13.3)-13.3

              Result: -3.5527136788005e-15

               

              So it looks like I'll have to do something like this to the result:

               

              myMod = a % b;

              if (Math.abs(myMod - b) < 0.00001) myMod = 0;

               

              ... which is the sort of thing that often needs to be done with floating

              point arithmetic, if I'm not mistaken.

               

              Thanks,

              Ariel

              • 4. Re: Funny modulus result? If this a rounding error?
                TᴀW Adobe Community Professional & MVP

                Ah, that function seems a bit better!

                 

                Thanks,

                Ariel

                • 5. Re: Funny modulus result? If this a rounding error?
                  Harbs. Level 6

                  You can also do something like this:

                   

                  (492.1*10) % (13.3*10);

                   

                  You can probably generalize this if need be...

                  • 6. Re: Funny modulus result? If this a rounding error?
                    TᴀW Adobe Community Professional & MVP

                    Hi Harbs,

                     

                    In this particular case I can't always know how many decimal places the

                    user will input.

                     

                    Thanks,

                    Ariel

                    • 7. Re: Funny modulus result? If this a rounding error?
                      Laubender Adobe Community Professional & MVP

                      Good to know!
                      It might come to my rescue right now that I'm working on a piece that involves modulo:

                       

                       

                      gcd(133,4921);
                      //Result: 133
                      
                      gcd(13.3,492.1);
                      //Result: 3.5527136788005e-15
                      
                      function gcd(c,d){
                          //http://pixelstech.net/article/1316635052_Gcd_Algorithm_with_JavaScript
                          if(d==0){return c;}else{return gcd(d,c%d)};
                          };
                      

                       

                       

                      Thank you all for bringing up this issue!

                       

                      Uwe

                      • 8. Re: Funny modulus result? If this a rounding error?
                        [Jongware] Most Valuable Participant

                        Arïel wrote:

                        What's going on? Is this because of floating-point arithmetic? In any  case, what can I do about it to get an accurate result?

                         

                        alert (492.1 % 13.3);

                        alert ((492.1 % 13.3) == 13.3);

                        alert (13.3 - (492.1 % 13.3));

                         

                        shows "13.3", "false" (!), and then the exact accurate-to-14 digits that Uwe gets. In this case it might just be on the threshold of floating point accuracy. I bet one of, or both numbers, cannot be accurately be represented in binary, just like you cannot accurately show "1/3" in decimal notation.

                        • 9. Re: Funny modulus result? If this a rounding error?
                          [Jongware] Most Valuable Participant

                          (Further thinking) If you still want to use the original mod, you need to test the epsilon fault tolerance for both endpoints. If you use some sort of rounding afterwards, you would get your original result "13.3" back, which clearly is wrong.

                           

                          I propose a dirty workaround to make it internally work with integers:

                           

                          alert (dirtyMod(492.1,13.3));
                          
                          function dirtyMod (a,b)
                          {
                                    var factor = 1;
                                   // while (!(Math.floor(a) == a && Math.floor(b) == b))
                                   // probably a faster coercion to integer:
                                    while (!((a>>0) == a && (b>>0) == b))
                                    {
                                              factor *= 10;
                                              a *= 10;
                                              b *= 10;
                                    }
                                    return (a % b)/factor;
                          }
                          

                           

                          Message was edited by: [Jongware]

                          • 11. Re: Funny modulus result? If this a rounding error?
                            TᴀW Adobe Community Professional & MVP

                            Update to my function:

                             

                            function safeMod(a,b){

                                 var myMod1 = a%b, myMod2 = Number(String(myMod1));

                                 Math.abs(myMod1 - myMod2)<.000001 && return 0;

                                 return myMod2;

                            }

                             

                            (just changed line 4)

                             

                            Ariel

                            • 12. Re: Funny modulus result? If this a rounding error?
                              TᴀW Adobe Community Professional & MVP

                              Sorry, I'm getting hopelessly confused. What I meant was something this:

                               

                              function safeMod(a,b){

                                   var myMod1 = a%b;

                                   Math.abs(myMod1 - b)<.000001 && return 0;

                                   return Number(String(myMod1));

                              }

                               

                              Ariel

                              • 13. Re: Funny modulus result? If this a rounding error?
                                Laubender Adobe Community Professional & MVP

                                @Arïel – are you sure with that function?

                                 

                                I'm getting an error message with line 3 of your function from the ESTK:
                                "illegal use of the reserved word "return"";

                                 

                                 

                                //USAGE:
                                $.writeln(safeMod(10,10));
                                //Expected result: 0
                                //Result: Error message "illegal use of the reserved word "return""
                                
                                
                                function safeMod(a,b){
                                     var myMod1 = a%b;
                                     Math.abs(myMod1 - b)<.000001 && return 0;
                                     return Number(String(myMod1));
                                };
                                

                                 

                                 

                                Uwe

                                • 14. Re: Funny modulus result? If this a rounding error?
                                  TᴀW Adobe Community Professional & MVP

                                  So much for my attempt to write like Marc Autret! Let's keep it simple then:

                                   

                                  function safeMod(a,b){
                                  var myMod1 = a%b;
                                  if(Math.abs(myMod1 - b)<.000001) return 0;
                                  return Number(String(myMod1));
                                  }; 

                                  Thanks,

                                  Ariel

                                  PS Thanks Trevor for pointing out that this post was blank. I have edited it manually.

                                  • 15. Re: Funny modulus result? If this a rounding error?
                                    Trevorׅ Adobe Community Professional

                                    Jongware,

                                     

                                    Nice function

                                    You just nead to add in the factor check so that the script doesn't go on endlessly with numbers like 1/3

                                    while (!((a>>0) == a && (b>>0) == b) && factor<100000000)

                                     

                                    Ariel,

                                     

                                    You still seem to be having thunderbird issues.

                                    Your last reply came out blank.

                                     

                                    Regards

                                     

                                    Trevor