6 Replies Latest reply on Apr 14, 2010 11:46 AM by xbytor2

    Math Confusion

    jugenjury Level 2

      I can't figure this one out to save my life. My document size is 2592X3888. With the following code:





      The alerts come out as







      The first and third alerts should come out as negative. They don't. If I change the code from w to w/4 and h to h/4, the alerts are as follows:







      Not only should the first and third come out positive, but the last 2 should be whole numbers.


      What am I missing?

        • 1. Re: Math Confusion
          Michael L Hale Level 5

          I think the results will depend on which version of Photoshop you are using and the ruler unit settings.


          If you are using CS4 and you want the w and h values in pixels you could do something like this




          • 2. Re: Math Confusion
            jugenjury Level 2

            Thanks again, Mike. I'm still confused why this happens. I do set the following also...


            app.preferences.rulerUnits = Units.PIXELS;
            app.preferences.typeUnits = TypeUnits.PIXELS;


            I thought that would put all measurements and units in pixels globally for the script. I do get the correct value, but the wrong sign. It seems that when the width and height values are before the operator (- in this case) the answers come out correct but when they are after the operator, they are incorrect. The .as('px') did work to fix that problem, though. I'm still curious why.


            Also, do you have any idea why I'm not getting a value of 28 for the last 2 lines?

            • 3. Re: Math Confusion
              Michael L Hale Level 5

              I think the first part has to do with how the different version of Photoshop deal with UnitValues. In the first code you post h and w are a UnitValue objects that defaults to the current ruler unit.


              I think the second problem is the as() method of UnitValue returns a number literal instead of a number object. And that sometimes makes a difference. Xbytor can explain it better that I can. But one fix would be to make sure h is a number object.


               alert( w instanceof Number );// false
               alert( w.constructor == Number );
               alert( typeof w );
              alert( h instanceof Number );// true
              • 4. Re: Math Confusion
                xbytor2 Level 4
                The .as('px') did work to fix that problem, though. I'm still curious why.


                UnitValue overloads the arithmetic operations. This means that Adobe has to provide new functions that do '+', '-', etc... with UnitValue objects as well as UnitValue objects and normal numbers. The implementation of these overloads has changed overtime, mostly for the better I would assume. However, since many of us here, myself included, must have our scripts run on multiple versions of PS, it would be inadvisable to rely on the overloads for anything.


                To get around this problem, always use UnitValue.as("px") to get a number value and use that in your calculations. If your version of PS doesn't have UnitValue.as(), make sure that your ruler units pref is set to pixels and access the UnitValue.value property.


                As to why you didn't get exactly -28, I would attribute it to some internal conversion round-off error in one of the UnitValue overload functions.

                1 person found this helpful
                • 5. Re: Math Confusion
                  jugenjury Level 2

                  Thanks, xbytor. This sorta makes sense to me. Well enough that I get the general idea.


                  I am using CS4, but I would like my script to also be compatible with earlier versions. I tried another workaround for it that seems to be ok.




                  Granted, this won't give a negative value when one is supposed to be there, but the script uses the absolute value of the number so that isn't a problem.


                  I was using this in a script another poster here requested to zoom in on a specific pixel. It doesn't quite work but I'm going to post it anyway in case anyone has an idea on where to go from there.

                  • 6. Re: Math Confusion
                    xbytor2 Level 4

                    Do this:




                    and you will be ok from PSCS on up and you'll avoid all of the problems you've been seeing. It won't work for PS7 but I doubt that will be a problem for you.