2 Replies Latest reply on Dec 3, 2010 7:19 PM by jkeener1

    Bizarre error adding Numbers in Flex 4

    jkeener1

      I'm having a bizarre error adding numbers in Flex 4.  In all cases, I'm getting the correct result, but sometimes I'm getting extra precision with a 5 at the end.

       

      For example,

       

      If I try to add 5 and 1.56, I get 6.5600000000000005

       

      a lot of numbers add correctly, it's just certain combinations that have the problem.

       

      Here is a small app that demonstrates the issue:

       

      <?xml version="1.0" encoding="utf-8"?>

      <s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"

                        xmlns:s="library://ns.adobe.com/flex/spark"

                        xmlns:mx="library://ns.adobe.com/flex/mx" minWidth="955" minHeight="600">

           <s:layout>

                <s:VerticalLayout/>

           </s:layout>

           

           <fx:Script>

                <![CDATA[

                     protected function button1_clickHandler(event:MouseEvent):void

                     {

                          var num1:Number = Number(number1.text);

                          var num2:Number = Number(number2.text);

                          

                          var result:Number = num1 + num2;

       

      //                    result = 1.56 + 5;

      //                    result = Number(1.56) + Number(5);

                          

                          resultLabel.text = String(result);

                     }

                ]]>

           </fx:Script>

           

           <s:TextInput id="number1"/>

           <s:TextInput id="number2"/>

           <s:Button label="Add Values" click="button1_clickHandler(event)"/>

           <s:Label id="resultLabel"/>

      </s:Application>

       

      Any assistance would be appreciated!

       

      Thanks!

       

      Jon Keener

      jkeener1@gmail.com

        • 1. Re: Bizarre error adding Numbers in Flex 4
          GordonSmith Level 4

          This question comes up all the time. It's a Fact Of life of modern microprocessors and computer languages, which implement floating-point arithmetic using binary fractions (1/2, 1/4, 1/8, etc.), not decimal fractions (1/10, 1/100, 1/1000, etc.), according to the IEEE-754 spec. Many decimal fractions, such as 1.56, cannot be exactly represented as a finite binary fraction. (Or the exact representation in binary might be finite but require more fractional digits that the spec allows.) Although your source program says "1.56", it gets compiled and executed as the IEEE-754 binary-fraction that is closest to 1.56, but it isn't exactly equal to 1.56, and then this slight error gets propogated through calculations, and the conversion back to decimal is again inexact.

           

          Generally, people just use methods like toPrecision() to round things off a bit and hide this problem. Or they go write or find a (much slower) ActionScript library that does true decimal floating-point arithmetic by storing decimal fractions rather than binary fractions.

           

          Gordon Smith

          Adobe Flex SDK Team

          1 person found this helpful
          • 2. Re: Bizarre error adding Numbers in Flex 4
            jkeener1 Level 1

            Thanks for the info!  I'll have to set the precision after the calc.