14 Replies Latest reply on Sep 23, 2008 4:24 PM by sstanleyau

    Getting rid of unwanted precision

    Roy McCoy Level 1
      I'm generally enjoying InDesign after many years of holding out with PageMaker, but one thing I haven't found convenient about it is its apparent tendency to give the user more precision than he wants, needs, or can use. If I want a tab at 7mm and try to place one there, for example, ID will give me one at 7.188 or something like that, and then I'll have to go into the dialog box and manually delete the unwanted fractional value.

      Now I'm starting with ID scripting, and right at the beginning, in one of the first specimen scripts of the ID scripting tutorial - namely "ImprovedHelloWorld.applescript" - I'm getting this in an apparently excessive degree, with for example 27.699999999946 and 17.999999999994 as results, rather than the normal, desirable, legible and manageable 27.7 and 18.

      Can anyone give me any advice on how to weed out this extraneous and unwanted precision, perhaps in the program at large as well as on the scripting side of it?

      Thanks,

      Roy McCoy
      Rotterdam, NL
      ID CS3, OS X 10.5.5, MacBook Pro
        • 1. Re: Getting rid of unwanted precision
          Level 1
          Roy,

          Use ESTK's Help menu to look at the Core JavaScript object model. Take a look at the Number.toString() method.

          Dave
          • 2. Re: Getting rid of unwanted precision
            Roy McCoy Level 1
            Hi Dave, it's great knowing you're here. I did open ESTK (for the first time ever - I wasn't even aware of it before), and I looked at all the Number Instance Elements. But I didn't see what you meant, and anyway I'm sticking with AppleScript for the time being, since I haven't found the time even to get back into that lately, after years of laziness just using my old stuff with PageMaker.

            Forgetting scripting for the moment, perhaps I can ask you if you've suffered the same general annoyance with all the multiple decimal places. To mention my previous example again, you would click on the tabs ruler in PageMaker and get a tab at an even or a half millimeter, and if you for some reason in an exceptional case wanted something more precise than that you could enter it manually. Is there any reason why it shouldn't be that way now? Or have I missed something, like you should click with the shift key or something if you want an even increment.

            Getting back to scripting, I'm sure I can do rounding in AppleScript as well as JavaScript, and I did it with PM scripting to get dimensions on graphics down to a level I would care to resize to in Photoshop. Looking back at the old scripts, ha, I see a "Dave Saunders" folder with a Height Calculating PM script from ten years ago. I also see "Folios Ahoy" and various other things. I guess we'll be in touch again if you're hanging out here and this is a main place for InDesign scripting, but it will be sad for me if you're mainly into JavaScript... though I suppose I could switch, as I've barely started.

            Roy McCoy
            Rotterdam, NL
            • 3. Re: Getting rid of unwanted precision
              Level 1
              The precision is a result of base 2 (binary) numbers and base 10 (decimal) numbers not mapping "precisely" on to each other. They're something you have to live with and accommodate in your scripts.

              Ah, I realize now that when I first saw your name on this topic, I mistook you for Roy Marshall who does use JavaScript.

              If you're comfortable with AppleScript then stay with it. I switched to JavaScript partly because I could never come to grips with AppleScript whose statements and partly because JavaScript is cross-platform.

              Nice to see you again (now I've worked out who you are!).

              Dave
              • 4. Re: Getting rid of unwanted precision
                [Jongware] Most Valuable Participant
                Also note that the measurements you see on the screen might be re-calculated from the internal measurements (I seem to recall ID uses points internally, but that hardly matters).

                The issue of a wanted "exact" precision might bite you back in some unexpected ways. For example, if you place an object at "2cm,2cm", it may appear in the document as "2.000001cm,2.000001cm" because of internal rounding. Now, if you want to check if it's still there some time later, hey, there is nothing at "2cm,2cm" (fortunately, there are other ways to identify objects
                i not
                based on their exact position).
                The same happens all the time with Word imported documents, where in the UI you can see a text indent value of "24pt". Well, that's already rounded, as you cannot
                i search
                for this 'exact' indent of 24pt -- checking with a script reveals it to be, for example, at 23.9947658(etc.).
                • 5. Re: Getting rid of unwanted precision
                  Peter Kahrel Adobe Community Professional & MVP
                  All the more reason for a tweak to the Find/Change dialog. You should be ablt to search for e.g. "indents smaller than 24 points", or "baseline shifts other than 2 points". Submit the requests!

                  Peter
                  • 6. Re: Getting rid of unwanted precision
                    Roy McCoy Level 1
                    A new message was posted by Dave Saunders in

                    InDesign Scripting --
                      Getting rid of unwanted precision

                    The precision is a result of base 2 (binary) numbers and base 10 (decimal) numbers not mapping "precisely" on to each other. They're something you have to live with and accommodate in your scripts.

                    I'm embarrassed to admit it, but after reading this I still continued to suspect that it might have something to do with using metric measurements rather than inches or picas and points. So I set up a US Letter page with measurements set to inches. Rather to my surprise, the behavior was precisely the same, with again strings of undesiredly overprecise values.

                    So I guess you're right, but why is it this way in InDesign and it wasn't in PageMaker? I don't think I could have missed it if it was that way in the earlier program, since though I wasn't really an expert PM scripter, I nonetheless had maybe hundreds of PM scripts and... well, okay, it did happen, I now re-remember, and I had to do some rounding (see my old Get and Log Dimensions/Crops PM script, below). But I think the precision was less excessive and obtrusive, like maybe two or three decimal places rather than four or five.
                    And it didn't do it with the tabs ruler -- that really continues to irk me. I finally concluded that it simply doesn't make sense to try to use the ruler when you know where you want to place your tab. Enough of this "But I don't want 4.0217, I want 4 on the nose" and then having to go and type it in -- better to just type "4"  in the first place.

                    Oh, now I finally see what it's doing. It's providing ruler tabs in points no matter what your measurement units are, and displaying the points precisely in millimeters or whatever. You can get even inches, half inches, and quarter inches, so there's some pro-inch discrimination after all. But what really drives me nuts about the excessive unwanted precision in the tab dialog is that not only is the value unwantedly precise, but you can't even see it. When I click on the ruler and get a tab at 34.572 mm, for example, I can't see the 34, just the .572 mm. Admittedly, hitting the up/down arrow keys rounds the tab setting right off and you can quickly adjust a tab using that. 

                    Ah, I realize now that when I first saw your name on this topic, I mistook you for Roy Marshall who does use JavaScript.

                    Oh yeah, Roy Marshall, I used to go out with his sister. :-)

                    If you're comfortable with AppleScript then stay with it. I switched to JavaScript partly because I could never come to grips with AppleScript whose statements and partly because JavaScript is cross-platform.

                    I wouldn't say I was comfortable with AppleScript, but I will stay with it at least for the time being. For one thing, the current version of QuicKeys is well integrated with it, and I'll probably want to script some non-Adobe apps.

                    Nice to see you again (now I've worked out who you are!).

                    Thanks and hyperlikewise.

                    Replies by email are OK.

                    Well, it still isn't Compuserve with MacNav, but at least you apparently can now follow a thread without having to go to the web.


                    Roy

                    *******

                    GET AND LOG DIMENSIONS/CROPS

                    -- Run on copy, not on original, but be sure to save latest version first.
                    -- Turn off facing pages before running.
                    -- If rounding off doesn't work, apply "Get and Log Cs/Cs Roundoff"
                    -- If "invalid state" alert, look for hidden layer and delete from copy
                    -- (but wasn't 05/02/04, when appeared to be because of grouping).
                    --This is an evolving script to automate the acquisition of dimensions of
                    --placed graphics. Use on copy, not on original, to avoid problems with
                    --facing-pages switch. Corrected to handle PICT and EPS
                    --as well as inline ( but kl 00 - ?). Loops to 500, generally unnecessarily
                    --so abort when it's found all periods and delete the one at the
                    --end along with unneeded graphics listings.

                    sub doittoit
                    if drawnum = -2
                      select lefttopx, lefttopy
                    else
                      select drawnum
                    endif
                    getlinkinfo >> filename, ...
                    filename = filename / ":"
                    --now create myar variable, which simply registers sides w/crops
                    myar = ""
                    getimageframe >> imageleft, imagetop, imageright, imagebottom
                    getcroprect >> cropleft, croptop, cropright, cropbottom
                    if not(imageleft = cropleft)
                       myar = myar + "left "
                    endif
                    if not(imagetop = croptop)
                      myar = myar + "top "
                    endif
                    if not(imageright = cropright)
                      myar = myar + "right "
                    endif
                    if not(imagebottom = cropbottom)
                      myar = myar + "bottom "
                    endif
                    if myar = ""
                    logappend " Great Royzito:linked/cropped images.log"
                    log "page " + quote(x) + ":" + ^(tab) + filename(len(filename)) + ", " + quote(zstrip((righttopx- lefttopx) / 10)) + " x " + quote(zstrip((leftboty - lefttopy) / 10)) + "; not cropped"
                    logend
                    return
                    else
                    logappend " Great Royzito:linked/cropped images.log"
                    log "page " + quote(x) + ":" + ^(tab) + filename(len(filename)) + ", " + quote(zstrip((righttopx- lefttopx) / 10)) + " x " + quote(zstrip((leftboty - lefttopy) / 10)) + "; cropped " + myar
                    -- Following line doesn't work, because device units rather than mms
                    -- + quote(zstrip((cropright-cropleft) / 10))
                    logend
                    endsub

                    sub incrementprevious1
                    textcursor +char, 1
                    delete
                    textselect -char, 1
                    gosub incrementprevious2
                    endsub

                    sub incrementprevious2
                    getstorytext 0, 0 >> theprevchar
                    switch theprevchar
                    caseof "0"
                    textenter "1"
                    break
                    caseof "1"
                    textenter "2"
                    break
                    caseof "2"
                    textenter "3"
                    break
                    caseof "3"
                    textenter "4"
                    break
                    caseof "4"
                    textenter "5"
                    break
                    caseof "5"
                    textenter "6"
                    break
                    caseof "6"
                    textenter "7"
                    break
                    caseof "7"
                    textenter "8"
                    break
                    caseof "8"
                    textenter "9"
                    break
                    caseof "9"
                    delete
                    textselect -char, 1
                    gosub incrementprevious2
                    break
                    default
                    endswitch
                    endsub

                    suppressautosave on
                    measureunits millimeters, millimeters, default
                    getpagenumbers >> startpage, numpages, ...
                    getpageoptions >> doublesided, facingpages
                    pageoptions doublesided, 0
                    logbegin " Great Royzito:linked/cropped images.log"
                    log "List of Linked/Cropped Images with Dimensions in Centimeters" + ^(cr)
                    logend
                    loop x = startpage, startpage + (numpages - 1)
                      page x
                      getobjectidlist >> numobjects, objectlist
                      deselect
                      loop n = 0, numobjects - 1
                        set objectlist >> objectid, maskid, groupid, drawnum, type, transformed, linked, lefttopx, lefttopy, rightbotx, rightboty, righttopx, righttopy, leftbotx, leftboty, objectlist
                    --now "doittoit" for all TIFFs (6), PICTs (8, 9) and EPS (11)
                        if type = 6
                          gosub doittoit
                        endif
                        if type = 8
                          gosub doittoit
                        endif
                        if type = 9
                          gosub doittoit
                        endif
                        if type = 11
                          gosub doittoit
                        endif
                      endloop
                    endloop
                    logappend " Great Royzito:linked/cropped images.log"
                    log "."
                    logend
                    deselect
                    new
                    import " Great Royzito:linked/cropped images.log"
                    place column 1 left, column top
                    printoptionsps 2, 0, 0, 1, 1, off, 0, 0, 0, 1
                    printpaperps "A4", "Lower Cassette", 297, 210, 0, 0, -2, 0, 0, 25, 0, 1000, 0
                    select 30m, 30m
                    try editstory
                    textcursor -story
                    FindParaAttr "Any", -3, -3
                    FindTypeAttr1 "Any", -3, -3, -3, -3, -3
                    ChangeTypeAttr1 "Any", -3, -3, -3, -3, -3
                    ChangeParaAttr "Any", -3, -3
                    --errorchecking on
                    --FIX SCRIPT HERE (?)
                    try changeall "0.99", "1.99"
                    try changeall "1.99", "2.99"
                    try changeall "2.99", "3.99"
                    try changeall "3.99", "4.99"
                    try changeall "4.99", "5.99"
                    try changeall "5.99", "6.99"
                    try changeall "6.99", "7.99"
                    try changeall "7.99", "8.99"
                    try changeall "8.99", "9.99"
                    try changeall "9.99", "OOPSIE-DAISY!!!"
                    textcursor -story
                    loop x=1, 500
                    try find ".", currentstory, stopatend, anycase, allinstances, clear
                    try findwindow close
                    --message "OK to here? Period selected?"
                    textcursor +char, 3
                    textselect +char, 1
                    getstorytext 0, 0 >> thechar
                    switch thechar
                    caseof "0"
                    textcursor +char, 1; delete
                    break
                    caseof "1"
                    textcursor +char, 1; delete
                    break
                    caseof "2"
                    textcursor +char, 1; delete
                    break
                    caseof "3"
                    textcursor +char, 1; delete
                    break
                    caseof "4"
                    textcursor +char, 1; delete
                    break
                    caseof "5"
                    gosub incrementprevious1
                    break
                    caseof "6"
                    gosub incrementprevious1
                    break
                    caseof "7"
                    gosub incrementprevious1
                    break
                    caseof "8"
                    gosub incrementprevious1
                    break
                    caseof "9"
                    gosub incrementprevious1
                    break
                    default
                    endswitch
                    endloop
                    delete
                    delete
                    textcursor -story, 1
                    find "OOPSIE-DAISY!!!"
                    --errorchecking off
                    suppressautosave off

                    • 7. Re: Getting rid of unwanted precision
                      Roy McCoy Level 1
                      A new message was posted by jongware in

                      InDesign Scripting --
                        Getting rid of unwanted precision

                      Also note that the measurements you see on the screen might be re-calculated from the internal measurements (I seem to recall ID uses points internally, but that hardly matters).

                      Right, that's what I just now noticed while replaying to Dave. Presumably the tab ruler isn't necessarily the only place ID does this.

                      The issue of a wanted "exact" precision might bite you back in some unexpected ways. For example, if you place an object at "2cm,2cm", it may appear in the document as "2.000001cm,2.000001cm" because of internal rounding. Now, if you want to check if it's still there some time later, hey, there is nothing at "2cm,2cm" (fortunately, there are other ways to identify objects not based on their exact position).
                      The same happens all the time with Word imported documents, where in the UI you can see a text indent value of "24pt". Well, that's already rounded, as you cannot search for this 'exact' indent of 24pt -- checking with a script reveals it to be, for example, at 23.9947658(etc.).

                      Thanks for the warning, and the confirmational "That's what ya gotta put up with."


                      Roy McCoy
                      Rotterdam, NL

                      • 8. Re: Getting rid of unwanted precision
                        sstanleyau Level 4
                        Roy, you keep calling it "precision", but in fact it's a representation of<br />the exact opposite: the fact that some base-10 numbers can't be represented<br />with precision as binary numbers.<br /><br />-- <br />Shane Stanley <sstanley@myriad-com.com.au>
                        • 9. Re: Getting rid of unwanted precision
                          Roy McCoy Level 1
                          On Sep 23, 2008, at 10:26 AM, Shane Stanley wrote:

                          Roy, you keep calling it "precision", but in fact it's a representation of the exact opposite: the fact that some base-10 numbers can't be represented with precision as binary numbers.

                          Well, I'll admit to not understanding that. Any base-10 integer can be represented as a binary number, right? I wouldn't immediately understand, then, why decimal values can't be similarly represented.

                          And in any event I still think it's unwanted precision, whatever's going on under the hood -- at least in the tabs dialog, even if you wouldn't want your scripts returning overly sloppy approximations. Doesn't anyone agree that it was a comparatively welcome convenience to be able to get a tab at a millimeter or half-millimeter interval, and to know that's what you were going to get when you clicked on the tabs ruler? Now, even if you're using the apparently preferred picas and points, you still have to work a lot more to get that same simple result, if you insist on continuing to try to do it with the ruler.

                          Shane Stanley, wow. I don't suppose you've gone over to JavaScript.


                          Thanks,

                          Roy McCoy
                          Rotterdam, NL

                          • 10. Re: Getting rid of unwanted precision
                            Level 1
                            Roy,

                            There are two issues here: what's presented in the interface and what goes on "under the hood." This is the scripting forum. By definition, scripts are dealing with "under the hood" data. So, even if the interface issues were all worked out to your satisfaction -- and it seems to me that one great leap forward would be for the text boxes to be large enough to accommodate the values -- in scripts, you should expect to see the unrounded values.

                            Dave
                            • 11. Re: Getting rid of unwanted precision
                              sstanleyau Level 4
                              On 23/9/08 10:46 PM, "Roy McCoy" <member@adobeforums.com> wrote:<br /><br />> Well, I'll admit to not understanding that. Any base-10 integer can be<br />> represented as a binary numb er, right? I wouldn't immediately understand,<br />> then, why decimal values can't be similarly represented.<br /><br />Think about how you represent the fraction 1/3 as a decimal -- you can't<br />*exactly*.<br /><br />> Shane Stanley, wow. I don't suppose you've gone over to JavaScript.<br /><br />No. I don't see the point.<br /><br />-- <br />Shane Stanley <sstanley@myriad-com.com.au>
                              • 12. Re: Getting rid of unwanted precision
                                Roy McCoy Level 1
                                On Sep 23, 2008, at 3:30 PM, Dave Saunders wrote:

                                There are two issues here: what's presented in the interface and what goes on "under the hood." This is the scripting forum. By definition, scripts are dealing with "under the hood" data. So, even if the interface issues were all worked out to your satisfaction -- and it seems to me that one great leap forward would be for the text boxes to be large enough to accommodate the values -- in scripts, you should expect to see the unrounded values.

                                Yeah, I suppose I simply wasn't generally getting such values in my PM scripts other than that "Get and Log Dimensions/Crops" one, or I would have noticed it more. But it was a different degree of precision. You got millimeters to three decimal places in scripts (which I rounded to two, the maximum in the PM interface and sufficient for my general DTP purposes). Now in ID three decimal places is standard in the interface (thus more precision than I want -- sorry), and you get twelve in script results. I'm still inclined to wonder whether anybody really missed the extra nine when they scripted PageMaker, and if not why there has to be so much apparent clutter with ID. But okay, I'll expect to see it nonetheless, thanks for the confirmation and the support on my gripe about the text boxes in the tabs dialog and perhaps elsewhere. My preferred solution for that, however, would be to go back to the clunky old PM precision of one or two decimal points or provide it as an option, rather than enlarging the boxes.


                                Roy McCoy

                                • 13. Re: Getting rid of unwanted precision
                                  Roy McCoy Level 1
                                  On Sep 24, 2008, at 1:24 AM, Shane Stanley wrote:

                                  Well, I'll admit to not understanding that. Any base-10 integer can be
                                  represented as a binary numb er, right? I wouldn't immediately
                                  understand, then, why decimal values can't be similarly represented.

                                  Think about how you represent the fraction 1/3 as a decimal -- you can't
                                  *exactly*.

                                  That's right -- but at this point, having learned that the extra decimal
                                  points are apparently unavoidable, I feel like Forrest Gump about it:
                                  "That's all I have to say about that."

                                  Shane Stanley, wow. I don't suppose you've gone over to JavaScript.

                                  No. I don't see the point.

                                  Yeah -- why encourage Windows?  :-)


                                  Thanks,

                                  Roy McCoy
                                  Rotterdam, NL

                                  • 14. Re: Getting rid of unwanted precision
                                    sstanleyau Level 4
                                    On 24/9/08 9:59 AM, "Roy McCoy" <member@adobeforums.com> wrote:<br /><br />> Yeah -- why encourage Windows?<br /><br />It's more the fact that I do a lot of scripts that talk to more than one app<br />-- js in near-useless for that because of the proprietary nature of the js<br />engine.<br /><br />-- <br />Shane Stanley <sstanley@myriad-com.com.au>