This content has been marked as final.
Show 2 replies

1. Re: simple calculation  subtraction
(Aandi_Inston) Aug 14, 2008 7:49 AM (in response to (Terry_Secor))Seems fairly typical rounding inaccuracies with floating point
arithmetic. If working with money it's vital to use only whole
numbers (e.g. work in cents): no accountant would accept money treated
as floating point.
Aandi Inston 
2. Re: simple calculation  subtraction
gkaiseril Aug 14, 2008 9:52 AM (in response to (Terry_Secor))Are the numbers you are subtracting calculated?
Because if I enter the values into form fields and perform the calculation, I come up with a different answer. The format changes the display of the value not the actual value. And you should also round the computed value. Remember the computer uses binary values for math computations and decimal values do not convert to finite binary decimals.
Unrounded values:
Text0 = 104175.38
Text1 = 7068.37
Text0  Text1 = 97107.01000000001
Rounded Values:
Text0 = 104175.38
Text1 = 7068.37
Text.0  Text.1 = 97107.01000000001
Rounded Difference:
Text0  Text1 = 97107.01
Since you will be using the rounding routine repeatedly, I would create a document level funciton to round a value to a given number of decimal places.
function Round(fValue, fPrecision) {
/*
Purpose: round a number to the requested number of decimal places
Inputs:
fValue = floating point number to be rounded
fPrecison = number of decimal places to be rounded to
Returns: rounded number
*/
// compute power of 10 for the number of decimal places
fAdjust = Math.pow(10, fPrecision);
// returned the value rounded to the requested number of decimal places
return Math.round(fValue * fAdjust) / fAdjust;
}
// Custom Calculation script using above function to demonstrate rounding
console.show();
console.clear();
var Text0 = this.getField("Text.0").value;
var Text1 = this.getField("Text.1").value;
console.println("Unrounded values:");
console.println("Text0 = " + Text0);
console.println("Text1 = " + Text1);
var Diff = Text0  Text1;
console.println("Text0  Text1 = " + Diff);
console.println("\nRounded Values:");
var Text0 = Round(Text0, 2)
var Text1 = Round(Text1, 2)
console.println("Text0 = " + Text0);
console.println("Text1 = " + Text1);
Diff = Text0  Text1;
console.println("Text0  Text1 = " + Diff);
console.println("\nRounded Difference:");
Diff = Round(Diff, 2);
console.println("Text0  Text1 = " + Diff);
event.value = this.getField("Text.0").value  this.getField("Text.1").value;