• Open Menu Close Menu
  • Apple
  • Shopping Bag
  • Apple
  • Mac
  • iPad
  • iPhone
  • Watch
  • TV
  • Music
  • Support
  • Search apple.com
  • Shopping Bag

Lists

Open Menu Close Menu
  • Terms and Conditions
  • Lists hosted on this site
  • Email the Postmaster
  • Tips for posting to public mailing lists
Re: a numeric bug.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: a numeric bug.


  • Subject: Re: a numeric bug.
  • From: has <email@hidden>
  • Date: Thu, 15 Sep 2005 09:05:26 +0100

deivy petrescu wrote:

>I don't get it. 10^23 is way below AS numeric limit (which I believe is over 10^300).

The precision limit is much lower, however - approx. ±9e15. Beyond that they're only approximations, and calculations involving the least significant digits are going to have inaccuracies.


>There is something very wrong with the way AS deals with numbers.

It's a limitation of the C doubles and math libraries used behind the scenes. You'll see the same precision error with Python floats, for example:

10.0 ** 23 % 7 # --> 1.0

unless you use a third-party module like gmpy <http://gmpy.sourceforge.net/ > or clnum <http://calcrpnpy.sourceforge.net/clnumManual.html> to give you arbitrary precision floats. Though integer math works ok out of the box as Python already includes a 'long' type for representing arbitrary size integers:

10 ** 23 % 7 # --> 5L

HTH

has
--
http://freespace.virgin.net/hamish.sanderson/
 _______________________________________________
Do not post admin requests to the list. They will be ignored.
Applescript-users mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:

This email sent to email@hidden

  • Follow-Ups:
    • Re: a numeric bug.
      • From: Emmanuel <email@hidden>
  • Prev by Date: RE: Converting Mac Roman and/or UTF 16 to UTF 8?
  • Next by Date: Re: Converting Mac Roman and/or UTF 16 to UTF 8?
  • Previous by thread: Re: a numeric bug.
  • Next by thread: Re: a numeric bug.
  • Index(es):
    • Date
    • Thread