15 Replies Latest reply on Mar 4, 2013 4:13 AM by Trevorׅ

# Funny modulus result? If this a rounding error?

Hi,

Strangely, I'm getting the following result in the ESTK Javascript console:

492.1 % 13.3

Result: 13.3

Clearly the result should be 0 (since 492.1 is a multiple of 13.3).

What's going on? Is this because of floating-point arithmetic? In any

case, what can I do about it to get an accurate result?

Thanks,

Ariel

• ###### 1. Re: Funny modulus result? If this a rounding error?

I suppose that technically it's an ExtendScript bug ... probably the '%' operator is deciding that 492.1 / 13.3 is 36 with a remainder of 13.3, instead of 37 with a remainder of 0.

But of course usually '%' is used with integers ... I wouldn't trust it with doing floating point arithmetic correctly anyway. In this particular case you could test for a remainder of either 0 or the divisor, but I'm not sure that I would trust that, either. I think that I'd need to know more about what you're trying to do before coming up with a reasonable suggestion.

Or of course you could do your own modulus operator, but I'm not sure I would trust even that:

function mod (a,b)

{

var m = ((a/b)-a)*b;

}

• ###### 2. Re: Funny modulus result? If this a rounding error?

Sorry, hit 'Send' unintentionally ... I think that mod(a,b) should be something more like:

((a/b)-Math.floor(a/b)) * b

I'm sure there are more elegant implementations ...

• ###### 3. Re: Funny modulus result? If this a rounding error?

I don't think it says anywhere that the % operator should only be used

with integers? At any rate, it seems to work fine with decimals in most

cases.

Your mod function seems rather strange to me: try a = 9, b = 3:

a/b = 3

3-9 = -6

-6 * 3 = -18

which clearly isn't the right answer for 9 % 3 ?

At any rate, I've meanwhile established that this is a floating-point

arithmetic error. Because, once again in the ESTK console, the following

shows that what looks like 13.3 isn't:

492.1 % 13.3

Result: 13.3

but....

(492.1%13.3)-13.3

Result: -3.5527136788005e-15

So it looks like I'll have to do something like this to the result:

myMod = a % b;

if (Math.abs(myMod - b) < 0.00001) myMod = 0;

... which is the sort of thing that often needs to be done with floating

point arithmetic, if I'm not mistaken.

Thanks,

Ariel

• ###### 4. Re: Funny modulus result? If this a rounding error?

Ah, that function seems a bit better!

Thanks,

Ariel

• ###### 5. Re: Funny modulus result? If this a rounding error?

You can also do something like this:

(492.1*10) % (13.3*10);

You can probably generalize this if need be...

• ###### 6. Re: Funny modulus result? If this a rounding error?

Hi Harbs,

In this particular case I can't always know how many decimal places the

user will input.

Thanks,

Ariel

• ###### 7. Re: Funny modulus result? If this a rounding error?

Good to know!
It might come to my rescue right now that I'm working on a piece that involves modulo:

```gcd(133,4921);
//Result: 133

gcd(13.3,492.1);
//Result: 3.5527136788005e-15

function gcd(c,d){
//http://pixelstech.net/article/1316635052_Gcd_Algorithm_with_JavaScript
if(d==0){return c;}else{return gcd(d,c%d)};
};
```

Thank you all for bringing up this issue!

Uwe

• ###### 8. Re: Funny modulus result? If this a rounding error?

Arïel wrote:

What's going on? Is this because of floating-point arithmetic? In any  case, what can I do about it to get an accurate result?

alert ((492.1 % 13.3) == 13.3);

alert (13.3 - (492.1 % 13.3));

shows "13.3", "false" (!), and then the exact accurate-to-14 digits that Uwe gets. In this case it might just be on the threshold of floating point accuracy. I bet one of, or both numbers, cannot be accurately be represented in binary, just like you cannot accurately show "1/3" in decimal notation.

• ###### 9. Re: Funny modulus result? If this a rounding error?

(Further thinking) If you still want to use the original mod, you need to test the epsilon fault tolerance for both endpoints. If you use some sort of rounding afterwards, you would get your original result "13.3" back, which clearly is wrong.

I propose a dirty workaround to make it internally work with integers:

```alert (dirtyMod(492.1,13.3));

function dirtyMod (a,b)
{
var factor = 1;
// while (!(Math.floor(a) == a && Math.floor(b) == b))
// probably a faster coercion to integer:
while (!((a>>0) == a && (b>>0) == b))
{
factor *= 10;
a *= 10;
b *= 10;
}
return (a % b)/factor;
}
```

Message was edited by: [Jongware]

• ###### 11. Re: Funny modulus result? If this a rounding error?

Update to my function:

function safeMod(a,b){

var myMod1 = a%b, myMod2 = Number(String(myMod1));

Math.abs(myMod1 - myMod2)<.000001 && return 0;

return myMod2;

}

(just changed line 4)

Ariel

• ###### 12. Re: Funny modulus result? If this a rounding error?

Sorry, I'm getting hopelessly confused. What I meant was something this:

function safeMod(a,b){

var myMod1 = a%b;

Math.abs(myMod1 - b)<.000001 && return 0;

return Number(String(myMod1));

}

Ariel

• ###### 13. Re: Funny modulus result? If this a rounding error?

@Arïel – are you sure with that function?

I'm getting an error message with line 3 of your function from the ESTK:
"illegal use of the reserved word "return"";

```//USAGE:
\$.writeln(safeMod(10,10));
//Expected result: 0
//Result: Error message "illegal use of the reserved word "return""

function safeMod(a,b){
var myMod1 = a%b;
Math.abs(myMod1 - b)<.000001 && return 0;
return Number(String(myMod1));
};
```

Uwe

• ###### 14. Re: Funny modulus result? If this a rounding error?

So much for my attempt to write like Marc Autret! Let's keep it simple then:

``function safeMod(a,b){ var myMod1 = a%b; if(Math.abs(myMod1 - b)<.000001) return 0; return Number(String(myMod1));};  ``

`Thanks,`

`Ariel`

`PS Thanks Trevor for pointing out that this post was blank. I have edited it manually.`

• ###### 15. Re: Funny modulus result? If this a rounding error?

Jongware,

Nice function

You just nead to add in the factor check so that the script doesn't go on endlessly with numbers like 1/3

while (!((a>>0) == a && (b>>0) == b) && factor<100000000)

Ariel,

You still seem to be having thunderbird issues.