Re: BIG number as string [re-round error]
Re: BIG number as string [re-round error]
- Subject: Re: BIG number as string [re-round error]
- From: julifos <email@hidden>
- Date: Thu, 19 Dec 2002 15:58:14 +0000
Ok.
I wasted all my morning in a routine which would return exact values for
numbers E+13, E+14 & E+15 (for E+15, would return "possible values", in a
maximum of 3 values), and now I deleted it, because I'm BIG IDIOT.
Anyway, I'll share my little keepsake with you:
Coercing a number equal or higher to E+13 to string will round it.
E+13: rounds last ten
E+14: rounds last hundred
E+15: rounds last thousand
E+16: rounds last ten-thousand
E+17: rounds last hundred-thousand
You can test this error easily:
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
set z to "999308635802549" as number
--> 9.99308635802549E+14
set y to (result as text) as number
--> 9.993086358025E+14
z - y
--> 49.0 (49 units error in a "E+14"!)
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
Additionally, any number E+15 will return allways an even number:
- if unit is {1,5,9}
* if ten is even, unit will round up to {2,6,0}
* if ten is odd, unit will round down to {0,4,8}
- if unit is {3,7}
* if ten is even, unit will round up to {2,6}
* if ten is odd, unit will round up to {4,8}
- if unit is {0,2,4,6,8} will return {0,2,4,6,8}
eg (UNTESTED):
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
set z to "9999308635802549" as number
--> 9.99308635802549E+14
Set x to BIGnumtostring(z)
--> "9999308635803000" (451 units error!)
UnRound(z,x)
--> may return (if I was not an IDIOT) this:
--> {"9999308635802550","9999308635802549","9999308635802548"}
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
Numbers E+16, E+17: untested. I think re-errors of coercing to string an
E+16 will be equal or less than 5 units (last unit is re-rounded).
.......
JJ
____
*E+15 numbers will throw this re-round error too in "display dialog"!
Yahoo! Mail Plus - Powerful. Affordable. Sign up now.
http://mailplus.yahoo.com
_______________________________________________
applescript-users mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/applescript-users
Do not post admin requests to the list. They will be ignored.