2 Replies Latest reply on Aug 14, 2008 9:52 AM by gkaiseril

    simple calculation - subtraction

      Why is it when I subtract 7,068.38 from 104,175.37 do I get 97,106.989999..
      I even tried to force it to work properly and it still does it. I tried writing
      Math.round(100*this.getField("Text#").value)/100 -
      Same answer?? This isn't that crucial but I'm just curious if anyone knows why it's calulating a simple subtraction formula that way? The rest of my from us calculating perfectly. Strange!
        • 1. Re: simple calculation - subtraction
          (Aandi_Inston) Level 1
          Seems fairly typical rounding inaccuracies with floating point
          arithmetic. If working with money it's vital to use only whole
          numbers (e.g. work in cents): no accountant would accept money treated
          as floating point.

          Aandi Inston
          • 2. Re: simple calculation - subtraction
            gkaiseril MVP & Adobe Community Professional
            Are the numbers you are subtracting calculated?

            Because if I enter the values into form fields and perform the calculation, I come up with a different answer. The format changes the display of the value not the actual value. And you should also round the computed value. Remember the computer uses binary values for math computations and decimal values do not convert to finite binary decimals.

            Unrounded values:
            Text0 = 104175.38
            Text1 = 7068.37
            Text0 - Text1 = 97107.01000000001

            Rounded Values:
            Text0 = 104175.38
            Text1 = 7068.37
            Text.0 - Text.1 = 97107.01000000001

            Rounded Difference:
            Text0 - Text1 = 97107.01

            Since you will be using the rounding routine repeatedly, I would create a document level funciton to round a value to a given number of decimal places.

            function Round(fValue, fPrecision) {
            Purpose: round a number to the requested number of decimal places

            fValue = floating point number to be rounded
            fPrecison = number of decimal places to be rounded to

            Returns: rounded number

            // compute power of 10 for the number of decimal places
            fAdjust = Math.pow(10, fPrecision);
            // returned the value rounded to the requested number of decimal places
            return Math.round(fValue * fAdjust) / fAdjust;

            // Custom Calculation script using above function to demonstrate rounding
            var Text0 = this.getField("Text.0").value;
            var Text1 = this.getField("Text.1").value;
            console.println("Unrounded values:");
            console.println("Text0 = " + Text0);
            console.println("Text1 = " + Text1);
            var Diff = Text0 - Text1;
            console.println("Text0 - Text1 = " + Diff);
            console.println("\nRounded Values:");
            var Text0 = Round(Text0, 2)
            var Text1 = Round(Text1, 2)
            console.println("Text0 = " + Text0);
            console.println("Text1 = " + Text1);
            Diff = Text0 - Text1;
            console.println("Text0 - Text1 = " + Diff);
            console.println("\nRounded Difference:");
            Diff = Round(Diff, 2);
            console.println("Text0 - Text1 = " + Diff);

            event.value = this.getField("Text.0").value - this.getField("Text.1").value;