11 Replies Latest reply on Jan 3, 2007 7:17 PM by Newsgroup_User

    Number precision

    dr.Evol
      Hi!

      How can I round the number to a particular decimal place?

      Example: 0.132456783929 should be 0.1324

      Any cool algorithms or existing solutions for that?

      Tnx!

      Cheers

        • 1. Number precision
          _Albertx_ Level 1
          Multiply the number by 10000 and then round the result, then divide it between 10000.

          Math.round(0.123456789 * 10000) / 10000;
          • 2. Re: Number precision
            kglad Adobe Community Professional & MVP
            yes, check this forum for a solution. (though you're truncation, not rounding.)

            this question is asked and answered approximately once per month.
            • 3. Re: Number precision
              kglad Adobe Community Professional & MVP
              that's not going to work, albert.

              first, he wants to truncate, not round (if the example is to be believed). (so, use Math.floor instead of Math.round.)

              second, you need to divide the result of your rounding/truncating by your factor.
              • 4. Re: Number precision
                _Albertx_ Level 1
                Well, He said, "round" altough in the example he typed a truncating number.

                And it will work, because I'm dividing the result. I've already done it.
                • 5. Re: Number precision
                  kglad Adobe Community Professional & MVP
                  you edited your message and corrected the 2nd error after i posted my comment. it wasn't corrected when i posted my comment.
                  • 6. Re: Number precision
                    dr.Evol Level 1
                    Tnx guys, will check it out...
                    Well, actually I need both - rounding and truncating. Im writting kind of financial calculations class, something like FV, PV etc methods...

                    Truncating could also be done simply by casting to string and cutting the lenght with substring...

                    Hmmm....
                    Math.round(0.123456789 * 10000) / 10000;

                    .. what if the multiplied number is still far below 0 ? (0.005) Then the round method will return zero...

                    • 7. Re: Number precision
                      _Albertx_ Level 1
                      Hahahahaha, 2 things...

                      dr.Evol.
                      In such case you have to count then numbers you want to round/trucate and multiply it for more or less "0"

                      and...

                      kglad
                      There was a confusion here, the problem is that I realized that I missed to divide the number before you write your second message. When you post your message i've already correct it.
                      That's because I think we are answering at the same time some questions :D
                      • 8. Re: Number precision
                        kglad Adobe Community Professional & MVP
                        use the appropriate factor. (actually, 10^5 is the wrong factor for your specific question. use 10^4.)

                        for currency formatting you might check canfieldstudios.com.

                        • 9. Re: Number precision
                          dr.Evol Level 1
                          tnx,

                          of course I can use different factors but the problem is that I dont know if the number will be in the range 10^-3,10^-4,10^-5 etc..
                          I guess I'll put a while loop to determine the range and according to that use a power of 10 to get the right factor.

                          Ok..guys..tell me...do you never sleep or you're in the wrong timezone ?! ;)

                          Good night and tnx!

                          cheerz
                          • 10. Re: Number precision
                            _Albertx_ Level 1
                            Jeje

                            Actually, I do.

                            I'm in GMT -6

                            Regards
                            • 11. Re: Number precision
                              Level 7
                              Please note .. MANY people use the newsgroup interface .. correction NEVER
                              appear there.

                              Please .. instead of (or as well as) editing in future, post a reply with
                              the corrected information.

                              Otherwise many people will not see your correction (eg. I cannot see it)
                              --
                              Jeckyl