Re: magic integer converter number 62?
Re: magic integer converter number 62?
- Subject: Re: magic integer converter number 62?
- From: David Reynolds <email@hidden>
- Date: Tue, 20 Feb 2001 01:42:52 -0800
on 2001.02.19 6:23 PM, John W Baxter at email@hidden said:
>
At 17:36 -0800 2/19/01, M Shaw wrote:
>
> Can anybody figure out why this works:
>
>
>
> set x to 62
>
> set b to ((x + (2.6 - (2.6 mod 1))) - x) as integer
>
>
>
> and this doesn't:
>
>
>
> set x to 61
>
> set b to ((x + (2.6 - (2.6 mod 1))) - x) as integer
>
>
My problem is that I can't immediately figure out why the first form
>
"works" (produces 2 rather than an error about not being able to make
>
[approximately] 2.0 into an integer).
>
>
Let's look at this smaller part of the puzzle:
>
>
set c to 2.6 - (2.6 mod 1)
>
c - 2
>
>
The result is not 0, it is the small non-zero number
>
5.77315972805081E-15
>
>
That's because 2.6 cannot be represented exactly as a binary floating point
>
number...neither can 0.6. The Mac is doing the best it can, but the values
>
2.6 and 0.6 don't exist...they have to be approximated.
I'm not sure about that, but I don't know enough about the low-level math
that AS uses...
I tested this and found two boundaries:
if the result is +64 or greater, it will work.
if the result is -65 or less, it will work.
So, the out of bounds for this is -64 through 63... 1 unsigned byte...
AHA! Since any larger float would use 2 unsigned bytes, more precision is
available, and it can be coerced...
...or something.
David
--
David Reynolds
<email@hidden>