I'm having a bizarre error adding numbers in Flex 4. In all cases, I'm getting the correct result, but sometimes I'm getting extra precision with a 5 at the end.
For example,
If I try to add 5 and 1.56, I get 6.5600000000000005
a lot of numbers add correctly, it's just certain combinations that have the problem.
Here is a small app that demonstrates the issue:
<?xml version="1.0" encoding="utf-8"?>
<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
xmlns:s="library://ns.adobe.com/flex/spark"
xmlns:mx="library://ns.adobe.com/flex/mx" minWidth="955" minHeight="600">
<s:layout>
<s:VerticalLayout/>
</s:layout>
<fx:Script>
<![CDATA[
protected function button1_clickHandler(event:MouseEvent):void
{
var num1:Number = Number(number1.text);
var num2:Number = Number(number2.text);
var result:Number = num1 + num2;
// result = 1.56 + 5;
// result = Number(1.56) + Number(5);
resultLabel.text = String(result);
}
]]>
</fx:Script>
<s:TextInput id="number1"/>
<s:TextInput id="number2"/>
<s:Button label="Add Values" click="button1_clickHandler(event)"/>
<s:Label id="resultLabel"/>
</s:Application>
Any assistance would be appreciated!
Thanks!
Jon Keener
This question comes up all the time. It's a Fact Of life of modern microprocessors and computer languages, which implement floating-point arithmetic using binary fractions (1/2, 1/4, 1/8, etc.), not decimal fractions (1/10, 1/100, 1/1000, etc.), according to the IEEE-754 spec. Many decimal fractions, such as 1.56, cannot be exactly represented as a finite binary fraction. (Or the exact representation in binary might be finite but require more fractional digits that the spec allows.) Although your source program says "1.56", it gets compiled and executed as the IEEE-754 binary-fraction that is closest to 1.56, but it isn't exactly equal to 1.56, and then this slight error gets propogated through calculations, and the conversion back to decimal is again inexact.
Generally, people just use methods like toPrecision() to round things off a bit and hide this problem. Or they go write or find a (much slower) ActionScript library that does true decimal floating-point arithmetic by storing decimal fractions rather than binary fractions.
Gordon Smith
Adobe Flex SDK Team
Thanks for the info! I'll have to set the precision after the calc.