the way numbers are represented on different platforms varies and it's not possible to store every possible number exactly.
the most accute value you can print out for that calculation in flex (at least without some custom high precision number class of which i have never come accross one) is
var var1:Number = 153682.62;
var var2:Number = -12266.240000000002;
var var3:Number = var1+var2;
trace(var1+var2); // 141416.38
trace(var3.toPrecision(21)); // 141416.380000000004657
trace(var3.toFixed(20)); // 141416.38000000000465661287
from the livedocs
In ActionScript, constants are defined with the maximum precision of double-precision IEEE-754 floating-point numbers
I've found a way to perhaps demonstrate the problem a little better
var var4:Number = -12266.240000000001;
trace(var4); // -12266.240000000002;
not all numbers can be represented exactly
thanks for reply,
You are right but there is some place where multiple value's sum is calcualte in for loop at that time i found something like .379999997 like this but if i calculate this value in some other way and i get 2 value one is plus and one is minus and then i sum then i will get result floating point like .38
i want both are same i don't know what can i do , because this issue occurs only for some value not for all value
can you post your code please, at least the part inculding the for loop where it works?
Actully original code is too long and value is comes from web services here i explain this 2 parts.
1 ) first display : Here there is amount of cateogory object and i calculate separate object as category wise as well as includeInFixedFees and notIncludeInFixed fees wise
so i will get 4 value.
IncludeInFixedFees : in that i get value of negative(-) amount and positive(+) amount of cateogry wise
NotIncludeInFixedFees : in that i get value of negative(-) amount and positive(+) amount of cateogry wise
using above value i create one list in that this negative value + positive value make total value;
when user click on total at that time this filter is apply on base list which condition is depends on cateogryid
this objects will pased in second display on gridData array.
2) second display : in this display have below loop
var intInExpTotal : Number = 0;
var intOutExpTotal : Number = 0;
for(var i:int=0; i<gridData.length; i++)
if(gridData[i].IncludedInFixedFee == true)
intInExpTotal += gridData[i].Amount;
trace(gridData[i].Amount +"...FD......SUM..."+ intInExpTotal);
intOutExpTotal += gridData[i].Amount;
trace(gridData[i].Amount +"...NFD......SUM..."+ intOutExpTotal);
here intInExpTotal / intOutExpTotal is total amount.
both the diaply have calulate total ammount but both are different some time when last when i calulate negative + postive value that time flex round up floating value on display 1 but not in display 2.
1 person found this helpful
I am unable to recreate your issue within a for loop without knowing where you are getting different values, and/or a simplified test case I can't really do much more for you except perhaps suggest you create your own PreciseNumber class.
Also consider what sort of object is your webservice storing the numbers as? And what numbers is it returning?
Thanks for reply ,
I think you are right now i will create new class to convert number and for multiplication and convert number i will use this class may be i will get help using some mathematical calculation.
But your answer is really helpful to me.