3 Replies Latest reply on Jul 1, 2008 5:01 PM by kglad

    Floating Point Arithmatic Error

    Michael_Anthony
      Hi,

      I know actionscript represents numbers and double precision floating point values. I'm having a problem where double arithmatic in actionscript doesn't match the results of the same double arithmatic in C++ / C#.

      EXAMPLE:

      In C++ / C#:
      ***************************************************
      double x, y, x1, y1;

      x = 209.4;
      y = 148.8;
      x1 = 203.0;
      y1 = 145.0;

      double ddx = x - x1;
      double ddy = y - y1;

      RESULT
      ddx: 6.4000000000000057
      ddy: 3.8000000000000114
      ***************************************************


      In Flash ActionScipt 2:
      ***************************************************
      var x, y, x1, y1;

      x = 209.4;
      y = 148.8;
      x1 = 203.0;
      y1 = 145.0;

      var ddx = x - x1;
      var ddy = y - y1;

      RESULT
      ddx: 6.39999999999992
      ddy: 3.80000000000024
      ***************************************************


      After researching Flash / Actionscript "var" stores numerical values as doubles ( 8 bytes ) just like doubles are stored in C++ / C# ( 8 bytes ). Why would there be a difference between the results of ddx and ddy? Are there different implementations of double floating point math? If so, Is there a way I can mimic the Flash / Actionscript version in C++ / C#?

      Any help would be great!

      Thanks!