Re: a numeric bug.
Re: a numeric bug.
- Subject: Re: a numeric bug.
- From: has <email@hidden>
- Date: Thu, 15 Sep 2005 09:05:26 +0100
deivy petrescu wrote:
>I don't get it. 10^23 is way below AS numeric limit (which I believe is over 10^300).
The precision limit is much lower, however - approx. ±9e15. Beyond that they're only approximations, and calculations involving the least significant digits are going to have inaccuracies.
>There is something very wrong with the way AS deals with numbers.
It's a limitation of the C doubles and math libraries used behind the scenes. You'll see the same precision error with Python floats, for example:
10.0 ** 23 % 7 # --> 1.0
unless you use a third-party module like gmpy <http://gmpy.sourceforge.net/ > or clnum <http://calcrpnpy.sourceforge.net/clnumManual.html> to give you arbitrary precision floats. Though integer math works ok out of the box as Python already includes a 'long' type for representing arbitrary size integers:
10 ** 23 % 7 # --> 5L
HTH
has
--
http://freespace.virgin.net/hamish.sanderson/
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Applescript-users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden