5 Replies Latest reply on Jul 21, 2006 6:12 PM by Newsgroup_User

    math.floor sucks?

    paullu
      copy this code in your layer as code:


      var tt:Number = Number("1002.36");
      trace("tt: "+Number(tt));

      trace("\n***********\n");

      trace(Math.floor(Number(1002.36)*Number(100)));
      trace(Math.floor(Number(tt)*Number(100)));

      and you will see:

      tt: 1002.36

      ***********

      100236
      100235

      I spend a lot of time to findout this problem in my project, this is just a resume of it.
        • 1. Re: math.floor sucks?
          kglad Adobe Community Professional & MVP
          that's not a problem with Math.floor. it's a problem with converting a string to a number and the fact that your computer uses binary arithmetic. so, what looks like 1002.36 to you, looks like 1002.3599999999 to your computer.

          use Math.round(), if you are able.
          • 2. Re: math.floor sucks?
            paullu Level 1
            but it's not , i just trace it before i used in math {look this line trace("tt: "+Number(tt));}


            you can try this:

            // cool numbers

            for(var i:Number = 0; i<1500; i++){
            for(var j:Number = 0; j<100; j++){
            var tt:Number = Number(String(i)+"."+String(j));
            //trace("tt: "+Number(tt));
            if((Number(tt)*Number(100)) != (Math.floor(Number(tt)*Number(100)))){
            trace((Number(tt)*Number(100))+" != "+(Math.floor(Number(tt)*Number(100)))+" original tt: "+tt);
            }
            }
            }


            the out put should be empty but it's not
            • 3. Re: math.floor sucks?
              Level 7
              paullu wrote:
              >
              > [...]
              >
              > I spend a lot of time to findout this problem in my project, this is just a
              > resume of it.

              This is not a problem with Math.floor. Instead you hit the known issue
              that numbers are not converted in the same way from string to binary
              representation between the flash authoring tool and the flash player.

              Typically:
              ==================
              // conversion from string to binary in the authoring tool (compilation):
              trace(1002.36 == 1002.36); // true

              // conversion from string to binary in the flash player (run time):
              trace(Number("1002.36") == Number("1002.36")); // true

              // comparison of the 2 conversions:
              trace(1002.36 == Number("1002.36")); // false
              ==================

              I've been told it isn't a bug but it's still a problem in my book, and
              one that's definitely worth knowing.

              Tim.
              • 4. Re: math.floor sucks?
                kglad Adobe Community Professional & MVP
                "but it's not...", what?
                • 5. Re: math.floor sucks?
                  Level 7
                  I've simplified the code a little .. no need for a lot of those 'Number'
                  calls .. and the full range generates a lot of output .. so try this

                  for (var i:Number = 1000; i<1100; i++) {
                  for (var j:Number = 10; j<100; j++) {
                  var tt:Number = Number(String(i)+"."+String(j));
                  if (tt*100 != Math.floor(tt*100)) {
                  trace(tt*100+" != "+Math.floor(tt*100));
                  }
                  }
                  }

                  The results are consistent with what kglad says .. converting something like
                  "1042.13" to a number does not give you an exact results .. sometimes it
                  will be slightly over, and sometimes slightly under and VERY rarely will it
                  be exact.

                  That is the nature of numbers and how they are stored in a computer.

                  So you've proved what kglad has said.
                  --
                  Jeckyl