I'm having a bizarre error adding numbers in Flex 4. In all cases, I'm getting the correct result, but sometimes I'm getting extra precision with a 5 at the end.
If I try to add 5 and 1.56, I get 6.5600000000000005
a lot of numbers add correctly, it's just certain combinations that have the problem.
Here is a small app that demonstrates the issue:
<?xml version="1.0" encoding="utf-8"?>
xmlns:mx="library://ns.adobe.com/flex/mx" minWidth="955" minHeight="600">
protected function button1_clickHandler(event:MouseEvent):void
var num1:Number = Number(number1.text);
var num2:Number = Number(number2.text);
var result:Number = num1 + num2;
// result = 1.56 + 5;
// result = Number(1.56) + Number(5);
resultLabel.text = String(result);
<s:Button label="Add Values" click="button1_clickHandler(event)"/>
Any assistance would be appreciated!
This question comes up all the time. It's a Fact Of life of modern microprocessors and computer languages, which implement floating-point arithmetic using binary fractions (1/2, 1/4, 1/8, etc.), not decimal fractions (1/10, 1/100, 1/1000, etc.), according to the IEEE-754 spec. Many decimal fractions, such as 1.56, cannot be exactly represented as a finite binary fraction. (Or the exact representation in binary might be finite but require more fractional digits that the spec allows.) Although your source program says "1.56", it gets compiled and executed as the IEEE-754 binary-fraction that is closest to 1.56, but it isn't exactly equal to 1.56, and then this slight error gets propogated through calculations, and the conversion back to decimal is again inexact.
Generally, people just use methods like toPrecision() to round things off a bit and hide this problem. Or they go write or find a (much slower) ActionScript library that does true decimal floating-point arithmetic by storing decimal fractions rather than binary fractions.
Adobe Flex SDK Team
Thanks for the info! I'll have to set the precision after the calc.