3 Replies Latest reply on Sep 9, 2011 2:43 PM by AlHolden

    Formatter rounds - the same number - two different ways?

    AlHolden Level 1

      See and/or run the 4.01 code below. Although the resulting number is the same, the formatted output is:

       

      8485.50

       

      8485.51

       

      Please advise what revision I should make so that I get 8485.51 in both cases.

       

      Thankew!!!

       

      <?xml version="1.0" encoding="utf-8"?>
      <s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
                      xmlns:s="library://ns.adobe.com/flex/spark"
                      xmlns:mx="library://ns.adobe.com/flex/mx" minWidth="200" minHeight="300">
          
           <fx:Declarations>

       

           <!-- comment in/out one or the other, the result is the same   
           <mx:CurrencyFormatter id="dolFmt"
                                     decimalSeparatorTo="."
                                     precision="2"
                                     currencySymbol=""
                                     rounding="nearest"
                                     useThousandsSeparator="false" />
           -->


           <mx:NumberFormatter id="dolFmt"
                                    decimalSeparatorTo="."
                                    rounding="nearest"
                                    precision="2"
                                    useThousandsSeparator="false" />


          </fx:Declarations>
          
           <fx:Script>
               <![CDATA[
                   [Bindable]
                   private var numString1:String = "8485.5050";
                  
                   [Bindable]
                   private var numStringA:String = "8349.0750";
                   [Bindable]
                   private var numStringB:String = "136.4300";
                  
                   // FYI: 8349.0750 + 136.4300 = 8485.505
                  
                   public function val(s:String):Number {
                       var n:Number = new Number;
                       if (isNaN(parseFloat(s))){
                           n = 0
                       } else {
                           n = parseFloat(s);
                       }
                       trace('val: ' + s + '=' + n);
                       return n;
                   }
               ]]>
           </fx:Script>
          
           <mx:VBox>
               <mx:Text text="{dolFmt.format( val(numString1) )}" />
               <mx:Text text="{dolFmt.format( val(numStringA) + val(numStringB) )}" />
           </mx:VBox>
          
      </s:Application>

        • 1. Re: Formatter rounds - the same number - two different ways?
          AlHolden Level 1

          I should add that the frequently blogged "floating point" solutions DO NOT appear to help.

           

          For example, this modification does not appear to change the results:

           

                      public function val(s:String):Number {
                          var n:Number = new Number;
                          if (isNaN(parseFloat(s))){
                              n = 0
                          } else {
                              var hack:Number = Math.pow(10, 5);
                              n = Math.round(hack * parseFloat(s)) / hack;
                          }
                          trace('val: ' + s + '=' + n);
                          return n;
                      }

          • 2. Re: Formatter rounds - the same number - two different ways?
            AlHolden Level 1

            Incorporating Josh's solution*  - after the simple addition step - seems to fix this example. I'm still hoping for something a little more elegant...

            * http://joshblog.net/2007/01/30/flash-floating-point-number-errors/

            Al

             

            <?xml version="1.0" encoding="utf-8"?>
            <s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
                           xmlns:s="library://ns.adobe.com/flex/spark"
                           xmlns:mx="library://ns.adobe.com/flex/mx" minWidth="200" minHeight="300" >
               
                <fx:Declarations>

             

                 <mx:NumberFormatter id="dolFmt"
                                         decimalSeparatorTo="."
                                         rounding="nearest"
                                         precision="2"
                                         useThousandsSeparator="false" />

             

                </fx:Declarations>
               
                <fx:Script>
                    <![CDATA[
                        [Bindable]
                        private var numString1:String = "8485.5050";
                       
                        [Bindable]
                        private var numStringA:String = "8349.0750";
                        [Bindable]
                        private var numStringB:String = "136.4300";
                       
                        // FYI: 8349.0750 + 136.4300 = 8485.505
                       
                        public function val(s:String):Number {
                            var n:Number = new Number;
                            if (isNaN(parseFloat(s))){
                                n = 0
                            } else {
                                n = parseFloat(s);
                            }
                            //trace('val: ' + s + '=' + n);
                            return n;
                        }
                       
                        public function correctFloatingPointError(number:Number, precision:int = 5):Number
                        {
                            var correction:Number = Math.pow(10, precision);
                            return Math.round(correction * number) / correction;
                        }

             

                    ]]>
                </fx:Script>
               
                <mx:VBox>
                    <mx:Text text="{dolFmt.format( val(numString1) )}" />
                    <mx:Text text="{dolFmt.format( correctFloatingPointError(val(numStringA) + val(numStringB)) )}" />
                </mx:VBox>
               
            </s:Application>

            • 3. Re: Formatter rounds - the same number - two different ways?
              AlHolden Level 1

              OK, I am STILL not in a better place here...

               

              Consider that this example produces a proper result only the first three times - and floating point solutions don't seem to help:

              5.51

              85.51

              485.51

              8485.50

               

              <?xml version="1.0" encoding="utf-8"?>
              <s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
                             xmlns:s="library://ns.adobe.com/flex/spark"
                             xmlns:mx="library://ns.adobe.com/flex/mx" minWidth="200" minHeight="300" >
                 
                  <fx:Declarations>
                   <mx:NumberFormatter id="dolFmt"
                                       decimalSeparatorTo="."
                                       rounding="nearest"
                                       precision="2"
                                       useThousandsSeparator="false" />
                  </fx:Declarations>
                 
                  <fx:Script>
                      <![CDATA[
                          [Bindable]
                          private var numString1:String = "5.5050";
                          [Bindable]
                          private var numString2:String = "85.5050";
                          [Bindable]
                          private var numString3:String = "485.5050";
                          [Bindable]
                          private var numString4:String = "8485.5050";
                      ]]>
                  </fx:Script>
                 
                  <mx:VBox>
                      <mx:Text text="{dolFmt.format( numString1 )}" />
                      <mx:Text text="{dolFmt.format( numString2 )}" />
                      <mx:Text text="{dolFmt.format( numString3 )}" />
                      <mx:Text text="{dolFmt.format( numString4 )}" />
                  </mx:VBox>
                 
              </s:Application>