6 Replies Latest reply on Jan 31, 2011 2:01 AM by Mohit M Chavda

    Differenet Result when Add two variable

    Mohit M Chavda Level 1

      Hi,

       

      Here i get 2 variable

       

          var var1:Number = 153682.62;
          var var2:Number = -12266.240000000002;

       

      when i do sum of this 2 variable i will get

       

      trace(var1+var2); // OUT PUT - > 141416.38

       

      but when i used calc and add this 2 number i will get 141416.379999999998

       

      Is any one help me to get this calculation that window calc is provided .

       


      Thanks,
      Mohit.

        • 1. Re: Differenet Result when Add two variable
          _spoboyle Level 4

          the way numbers are represented on different platforms varies and it's not possible to store every possible number exactly.

           

          the most accute value you can print out for that calculation in flex (at least without some custom high precision number class of which i have never come accross one) is

           

          var var1:Number = 153682.62;
          var var2:Number = -12266.240000000002;
                         
          var var3:Number = var1+var2;
          trace(var1+var2); // 141416.38

          trace(var3.toPrecision(21)); // 141416.380000000004657

          trace(var3.toFixed(20)); // 141416.38000000000465661287

           

          from the livedocs

               In ActionScript, constants are defined with the maximum precision of double-precision IEEE-754 floating-point numbers

           

          Edit:

           

          I've found a way to perhaps demonstrate the problem a little better

          var var4:Number = -12266.240000000001;

          trace(var4); // -12266.240000000002;

           

          not all numbers can be represented exactly

          • 2. Re: Differenet Result when Add two variable
            Mohit M Chavda Level 1

            hi,

             

            thanks for reply,

             

            You are right but there is some place where multiple value's sum is calcualte in for loop at that time i found something like .379999997 like this but if i calculate this value in some other way and i get 2 value one is plus and one is minus and then i sum then i will get result floating point  like .38

             

            i want both are same i don't know what can i do  , because this issue occurs only for some value not for all value

            • 3. Re: Differenet Result when Add two variable
              _spoboyle Level 4

              can you post your code please, at least the part inculding the for loop where it works?

              • 4. Re: Differenet Result when Add two variable
                Mohit M Chavda Level 1

                Hi,

                 

                Actully original code is too long and value is comes from web services here i explain this 2 parts.

                 

                1 ) first display : Here there is amount of cateogory object and i calculate separate object as category wise as well as includeInFixedFees and notIncludeInFixed fees wise

                 

                so i will get 4 value.

                 

                IncludeInFixedFees : in that i get value of negative(-) amount and positive(+) amount of cateogry wise

                NotIncludeInFixedFees : in that i get value of negative(-) amount and  positive(+) amount of cateogry wise

                 

                using above value i create one list in that this negative value + positive value make total value;

                 

                when user click on total at that time this filter is apply on base list which condition is depends on   cateogryid

                this objects will pased in second display on gridData array.

                 

                2) second display : in this display have below loop

                 

                               var intInExpTotal : Number = 0;
                                var intOutExpTotal : Number = 0;
                                for(var i:int=0; i<gridData.length; i++)
                                {
                                    if(gridData[i].IncludedInFixedFee == true)
                                    {
                                        intInExpTotal += gridData[i].Amount;
                                        trace(gridData[i].Amount +"...FD......SUM..."+ intInExpTotal);
                                    }
                                    else
                                    {
                                        intOutExpTotal += gridData[i].Amount;
                                        trace(gridData[i].Amount +"...NFD......SUM..."+ intOutExpTotal);
                                    }               
                                }  

                 

                     here intInExpTotal / intOutExpTotal is total amount.

                 

                 

                both the diaply have calulate total ammount but both are different some time when last when i calulate negative + postive value that time flex round up floating value on display 1 but not in display 2.

                • 5. Re: Differenet Result when Add two variable
                  _spoboyle Level 4

                  I am unable to recreate your issue within a for loop without knowing where you are getting different values, and/or a simplified test case I can't really do much more for you except perhaps suggest you create your own PreciseNumber class.

                   

                  Also consider what sort of object is your webservice storing the numbers as? And what numbers is it returning?

                  1 person found this helpful
                  • 6. Re: Differenet Result when Add two variable
                    Mohit M Chavda Level 1

                    Hi,

                     

                    Thanks for reply ,

                     

                    I think you are right now i will create new class to convert number and for multiplication and convert number i will use this class may be i will get help using some mathematical calculation.

                     

                    But your answer is really helpful to me.

                     

                    Thanks,
                    Mohit.