re: integers which just won't be round
re: integers which just won't be round
- Subject: re: integers which just won't be round
- From: Emmanuel <email@hidden>
- Date: Tue, 30 Apr 2002 18:04:18 +0200
At 5:28 PM +0200 29/4/02, Charles Arthur wrote:
>
Run in SE, the numbers come up as expected, with 1 dp.
>
Run in AStudio, I get stuff like 80.09999999999.
It is a widely spread habit (though, not a universal one) that "real"
numbers be coded according to how they are written in base 2. This is an
optimal coding, considering that most computers use 2-states devices (thus,
coding respectively for "0" and "1", the two digits in base 2).
For instance, the integer 13 = 8 + 4 + (not 2) + 1 gets coded as:
1 1 0 1
For instance, the real 3 = 2 + 1 *or* = 2 + 0.5 + 0.25 + 0.125 + etc can be
coded as:
1 1 . 0 0 0 0 0 0 etc
or as:
1 0 . 1 1 1 1 1 1 etc
I have assumed that the decimal separator has a fixed position, for
simplicity. Actually the coding is a bit more complex (so that it can
accomodate very big and very small numbers).
Of course, the number or bits used to encode a real number is finite (for
instance, 32 bits), so the number may get truncated:
1 0 . 1 1 1 1 1 1
really codes for 2.984375 instead of 3.
If some software (with my example) displays the numbers as "x.y" then it
will display "3.0". While if some other software is set to display numbers
as "x.yz", it will display the same number as "2.98" instead.
Emmanuel
_______________________________________________
applescript-users mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/applescript-users
Do not post admin requests to the list. They will be ignored.