5 Replies Latest reply on May 5, 2006 6:29 PM by Newsgroup_User

# addition problem - is actionscript/flash able to calculate correctly????

Hi,
I am adding two float numbers that return an incorrect result. Here is what I am doing...

var simpleMath:Number = Number(parseFloat("110.50")) - Number(parseFloat("107.10"));

The result I get is... Simple Math=3.40000000000001

How is this possible?
I have tried...
var simpleMath:Number = Number(110.50) - Number(107.10);

var simpleMath:Number = 110.50 - 07.10;

var numba:Number = 110.50;
var numbb:Number = 107.10;
var simpleMath:Number = numba - numbb;

They all give the same result...

When I try this:
Final_Payment.text = Number(Total_Payment_Due.text) - ((Number(Payment_Count.text) - 1) * Number(Payment_Amount.text));
Where Total_Payment_Due.text = 110.50 and Payment_Count.text = 36 and Payment_Amount.text = 3.06
I get this result: 3.39999999999999...

Can someone please tell me what I am going wrong. Or is there problem with actionscript/flash...

I am using Cold Fusion 7.0.1 and actionscript inside a Flash Form savecontent variable.

This is driving me crazy...

Also, is there a round function for floats? I know about the Math.round() but it only works on integers.

Any help would be extreemly appreciated!
Ed
• ###### 1. Re: addition problem - is actionscript/flash able to calculate correctly????
"ed_skinner" <webforumsuser@macromedia.com> wrote in message news:e3dcfr\$irm\$1@forums.macromedia.com...
>
> Hi,
> I am adding two float numbers that return an incorrect result. Here is
> what I am doing...
>
> var simpleMath:Number = Number(parseFloat("110.50")) -
> Number(parseFloat("107.10"));
>
> The result I get is... Simple Math=3.40000000000001
>
> How is this possible?

It's the same old problem we have always had with floating point stored in computer memory.. not just Flash. For dealing with
dollars and cents you can make it easy on yourself by multiplying both sides by 100 and rounding to eliminate decimals, then in
final calculation divide by 100.
You can also multiply/divide by 1000 if you need higher precision in the pennies.
tralfaz

• ###### 2. Re: addition problem - is actionscript/flash able to calculate correctly????
That is why most commercial programs store currency as whole numbers of
cents, rather than as floating point numbers (which by their very nature
cannot be exact)

You need to change your program to take the nature of numbers in computers
into account to avoid errors. The bug is really in how you are doing your
calculations, not in Flash, and the solution also lies with you.
--
Jeckyl

• ###### 3. Re: addition problem - is actionscript/flash able to calculate correctly????
whoa, there's jedi in the jeckyl. who knew?
• ###### 4. Re: addition problem - is actionscript/flash able to calculate correctly????
Hey Thanks guys. I used the technique of *100, calculate, round and then /100 to get it to work right. One might think that in all the advancement in computers that floating point numbers would be handled correctly by now!
Oh well, past another problem with the help of fellow programmers.
Folks... and may the force be with you!
• ###### 5. Re: addition problem - is actionscript/flash able to calculate correctly????
It doesn't matter how advanced a computer it is .. 0.1 (for example) can not
be expressed exactly in binary (which computers use) .. no matter what you
do, it is always only an approximation (a bit like 1/3 =
0.3333333333333333.... in decimal). Its just the mathematics of it.

Fixed point arithmetic (where you implicitly multiply by some number.. like
multiply by 100 for currency amounts to give you cents) is often done. For
instance, the x and y coordinates in Flash are internally fixed point
numbers (multiples of 1/20 = 0.05)
--
Jeckyl