This content has been marked as final.
Show 14 replies

1. Re: Integer math just doesn't add up
Ansury Dec 8, 2008 2:07 PM (in response to Boxy_999)Not 100% sure but I'd bet that the math is being done by default as a Number type, which is like a float in C/Java  so no dropping of the remainders. If you don't drop the remainders you get almost 8 (7.888...), which then gets rounded down (drop the entire remainder for "cpu style" integer rounding).... to 7.
I haven't done a heck of alot of math in Flex/Flash so I can't say what the default type it uses if none is specified, but I think it's Number. 
2. Re: Integer math just doesn't add up
Ansury Dec 8, 2008 2:11 PM (in response to Boxy_999)Wait wait, the debugger is saying 6? Heh. That seems strange, I'll have to check into that. I'd think you should get 7 there too.. 
3. Integer math just doesn't add up
Ansury Dec 8, 2008 2:17 PM (in response to Boxy_999)My debugger says 7, of course I'm on a Flex 2 workstation now, not FB3. Can't test 3 atm, maybe someone else can. Maybe it's a bug in the debugger? Kinda surprised that's possible, though. It's as if the debugger is using "int" instead of Number by default to do that math, or at least rounding down after each operation. 
4. Re: Integer math just doesn't add up
rtalton Dec 8, 2008 2:26 PM (in response to Boxy_999)Hi,
I'm no math major, but I get a result of 7.8888...
How are you arriving at an answer of 6? 
5. Re: Integer math just doesn't add up
Boxy_999 Dec 8, 2008 2:31 PM (in response to Boxy_999)Follow up:
I've broken the computation down into seperate elements. See example code below.
Using this approach produces the correct result : 6
It's not an elegant solution but it works. This suggests to me that the underlying code isn't evaluating a complex computation even with the aid of brackets.

6. Re: Integer math just doesn't add up
Boxy_999 Dec 8, 2008 2:39 PM (in response to rtalton)Hi rtalton,
Using integer math:
71 / 9 = 7
7 / 3 = 2
2 * 3 = 6
Regards 
7. Re: Integer math just doesn't add up
Ansury Dec 8, 2008 2:40 PM (in response to Boxy_999)Well, doing it that way, you're converting your answer into a uint each time instead of leaving it a Number for the next computation. That effectively rounds down the remainder, and there you go.
This doesn't show up in other languages typically because an integer type is usually the default type. 
8. Re: Integer math just doesn't add up
Ansury Dec 8, 2008 2:43 PM (in response to rtalton)quote:
Originally posted by: rtalton
Hi,
I'm no math major, but I get a result of 7.8888...
How are you arriving at an answer of 6?
Rounding down after EACH operation (computers like to always round down when doing integer math instead of using the .5 rule). 
9. Re: Integer math just doesn't add up
rtalton Dec 8, 2008 2:43 PM (in response to Boxy_999)I did a little testing, and what Flex is doing is rounding the result of each part of your formula, resulting in rounding errors.
71/9=7.888 (rounded to 7)
7/3=2.333 (rounded to 2)
2*3=6
If you want the result of six, you need to round each of your results, like this:
var x:int = 71/9;
var y:int = x/3;
var z:int = y*3; (z = 6)
Don't know why you'd want to do that, though. 
10. Re: Integer math just doesn't add up
Boxy_999 Dec 8, 2008 2:51 PM (in response to Ansury)Hi Ansury,
I agree with you that it would seem that the underlying code is using a Number type rather than Integer math. I've substituted Number for int/uint and seen the 7.888 result mentioned in other replies. I suspect this should be regarded as a bug simply because the difference between the desired result: 6 and 7.888 could in some situations be enormous. Out of interest, I have checked the computation in Python and it returns the correct result.
Regards 
11. Re: Integer math just doesn't add up
Boxy_999 Dec 8, 2008 3:19 PM (in response to rtalton)Hi rtalton,
This is one element of a computation being used in a games program where based on a known position of a cell in a grid (71 in this example) it's possible to calculate the grid position of another key cell. Integer math is required and other languages cope with it  certainly C, C++, Python, SQL but not Flex or should I be referring to it as ActionScript. It's not a criticism, it's just a little awkward but I can work around it.
Thanks for your responses.
All the best,
Simon 
12. Integer math just doesn't add up
rtalton Dec 8, 2008 6:44 PM (in response to Boxy_999)Actionscript handles it as long as the vars are declared first and it knows that you want integer results. AS is using strong typing, and all your vars should be typed. So, you just have to declare three vars to get it typed right. That may be different from other languages, I don't know. All I know is that I would want the exact number, not rounded, in most computations I'd be doing. Think payroll: hours, wages, etc. We sure wouldn't our numbers rounded down there now, would we? ;) 
13. Re: Integer math just doesn't add up
Boxy_999 Dec 9, 2008 7:34 AM (in response to rtalton)Hi rtalton,
I agree with everything you say but to get familiar with Flex I'm adapting a Suduko solver program that I've written in another language and the math is all integer based.
For example:
From a known cell position (cellPos) in a 9x9 cell grid...
To find x,y coords:
y=int(cellPos/9)
x=int(cellpos%9)
To find out which 3x3 cell region contains the known cell is:
int((((cellPos / 9) /3 ) * 3)) + int(((cellPos % 9) / 3))
The last equation should resolve to a number between between 0 and 8 (and it does in other languages) but I think the last time I tried the equation in a single statement in Flex it returned 10!!!
But now that I have a workaround using Flex everything is OK.
I really like to thank you and Ansury for your assistance in resolving this issue.
Best Regards. 
14. Re: Integer math just doesn't add up
Boxy_999 Dec 9, 2008 10:57 AM (in response to Boxy_999)I've got it working!!!!
This really is the last post.
I've been experimenting with brackets and I've been far too liberal with them (in some respects).
The following syntax works properly: