6 Replies Latest reply on Jun 11, 2009 5:30 PM by Randomsen

# Applescript problem with floating number (page item bounding values).

I have discovered this weird problem yesterday...

I'm moving a page item using bounding box value of another item. Say, using y1 (top) + a offset value (ex. 3). [in pixel units]

Later on i'm doing a distance compare on both object and applescript doesnt return correct value.

By Ex. (not applescript, in pseudo code)
Set tValue to (y1 of page item 1 > (y1 of page item 2 -3))

tValue is true while it should be false.

I have compared the value (floating number) and they are exactly the same (by exemple 10.12343837753).

I suspect something very low level (precision of number), but i dont have much more knowledge of the problems. I was able to find a few thread on this on other forums but there isnt much details.

The solution was to round the value, but it's quite inconvenient (applescript being really verbose).

Any idea of the why and if i could have handled the problem differently??
• ###### 1. Re: Applescript problem with floating number (page item bounding values).
There's no simple answer -- conversion between decimal numbers and binary<br />representations isn't exact (in any language). The only thing I can suggest<br />in your case is to try using 2.99999999 instead of 3, to see if that covers<br />you.<br /><br />-- <br />Shane Stanley <sstanley@myriad-com.com.au>
• ###### 2. Re: Applescript problem with floating number (page item bounding values).
Maybe i havent explained well the problem... the calculation *adding a offset* just add more info to the problem wich is not necessary.

I was doing a comparison today with the absolute vertical scale and absolute horizontal scale of a image (to check if it was proportionally scaled).

The get a reliable equality comparison, i had to use "as string" on the numbers (number1 = number2 report false otherwise??) and use of a mix of multiplication on the numbers and trunc to be able to evaluate the difference more precisely (there is many case where the proportionality is off by 0.0000001 by exemple).

It seam that Applescript is able to get the value of a number with the precision it has in Indesign but is not able to operate on it adequately. The problem also exist in user interface as the value reported to the user are not always representative of the value that is available "under the hood value". This bring some problems that are not always easy to understand (we had a very similar case in a Illustrator to Flash asset export scenario).

There is surely something logical that can explain this but i was not yet able to put my finger on it.. yet!
• ###### 3. Re: Applescript problem with floating number (page item bounding values).
The question is whether you wish to evaluate equality of [1] strings (the way a measurement is displayed in the UI of Indesign in this case) or [2] the 'equality' of two numbers.

For [1] you need decent support from the application at hand (I do not know for sure in the case of Indesign, but my guess is that it will 'fail' unavoidably at least some of the time; just think of the implications if a user switches the units from mm to point...)

For [2] because of all kinds of (unavoidable) rounding errors it hardly ever makes sense to compare two real numbers directly for equality, rather you may want to consider to compare the difference between two numbers with a very small value (i.e. a value that is very close to zero). If the difference is 'less than' that small value, you can consider the two numbers 'equal' otherwise they are 'different'.

HTH.

Olaf
• ###### 4. Re: Applescript problem with floating number (page item bounding values).
Just to be sure we are talking about the same thing, i have created a sample script that i think illustrate the problem i have.

If you run test(200.6543276) and test(200.1234567) you should see the difference in result.

I remember that in lingo there was a way to change the floating precision that was handled by the scripting engine with number but i was not able to find this with Applescript (even if i think having seen something like that somewhere a long time ago).

There is not alot of comments but it's quite a simple script, if any of you want to give it a try for fun and is missing information or get different result, post about it! :)

Eric

ps: I dont set the back the units either (usually i save the current config and set them back at the end of the script).

on run {}
-- 200.6543276=equal? 200.1234567= not equal?
--test(200.1234567)
test(200.6543276)
end run

on test(pX2Value)
try

make new document

tell document 1

set horizontal measurement units of view preferences to points
set vertical measurement units of view preferences to points

set tNewObj_A to make new text frame at beginning of page 1

set geometric bounds of tNewObj_A to {0.0, 10.0, 100.0, pX2Value}

set tNewObj_A_x1 to item 2 of (geometric bounds of tNewObj_A)
set tNewObj_A_y1 to item 1 of (geometric bounds of tNewObj_A)
set tNewObj_A_x2 to item 4 of (geometric bounds of tNewObj_A)
set tNewObj_A_y2 to item 3 of (geometric bounds of tNewObj_A)

set tNewObj_B to make new text frame at beginning of page 1

-- Change the geometry so that the new box B is aligned at right of box A, same height and width of 100.
set geometric bounds of tNewObj_B to {tNewObj_A_y1, tNewObj_A_x2, tNewObj_A_y2, tNewObj_A_x2 + 100}

-- Get the info on the geometry
set tNewObj_B_x1 to item 2 of (geometric bounds of tNewObj_B)
set tNewObj_B_y1 to item 1 of (geometric bounds of tNewObj_B)
set tNewObj_B_x2 to item 4 of (geometric bounds of tNewObj_B)
set tNewObj_B_y2 to item 3 of (geometric bounds of tNewObj_B)

-- Display value of tNewObj_A_x2 and tNewObj_B_x1
display dialog "tNewObj_A_x2:" & (tNewObj_A_x2 as string) & ", tNewObj_B_x1:" & (tNewObj_B_x1 as string)

-- check equals:
if tNewObj_A_x2 = tNewObj_B_x1 then
display dialog "tNewObj_A_x2 = tNewObj_B_x1"
set tDifference to tNewObj_A_x2 - tNewObj_B_x1

display dialog "Difference:" & (tDifference as string)
else

if tNewObj_A_x2 as string = tNewObj_B_x1 as string then
display dialog "Not equal as number, but equal as string."
else
display dialog "problem: not equal (as string or number)"
end if

-- Substraction in applescript... wich precision it handles??
set tDifference to tNewObj_A_x2 - tNewObj_B_x1

display dialog "Difference:" & (tDifference as string)
end if
end tell
end tell

on error pmsg
display dialog pmsg
end try
end test
• ###### 5. Re: Applescript problem with floating number (page item bounding values).
There is part of the code that is interpretated as a smiley... the code is tNewObj_B + parenthesis.
• ###### 6. Re: Applescript problem with floating number (page item bounding values).

I've encountered the same problem when evaluating the horizontal offset of characters, where they are shown to be the same, but failed to produce the boolean value it should have (true).

If you're trying to test the values to see if they are the same, set the values into strings, then compare them, and that has consistently worked for me, not to mention that rather than rounding, you can parse them back into real number  values.