Re: 1.0 divided by 10 = 0.10000000000000001?
Re: 1.0 divided by 10 = 0.10000000000000001?
- Subject: Re: 1.0 divided by 10 = 0.10000000000000001?
- From: Derrick Bass <email@hidden>
- Date: Tue, 28 Mar 2006 19:42:46 -0600
Folks
If there is a documentation bug here, or even if it isn't clear,
then please file a bug.
The documentation quite clearly says NSDecimalNumber is base-10.
We're trying to determine is whether or not that is correct. Marcel,
based on (what I think is a pretty inconclusive) comment from Chris
Kane, thinks yes (NSDecimalNumber is base-2), while I see no reason
to doubt the docs based on what he's presented.
Since both Marcel and I are both using the fallacy of appealing to
authority (Chris Kane is his case, the Apple documentation in mine),
I'd thought I'd throw in a little evidence.
I ported the first portion of Kahan's famous (another appeal to
authority) "paranoia" program (see, e.g. <http://
orion.math.iastate.edu/burkardt/c_src/paranoia/paranoia.html>) to
test NSDecimalNumber. It confirms that it is base-10, although the
precision is less than the documentation says (34 digits; the docs
say 38).
I'm enclosing my port so anyone who's interested can check to be sure
I did it right.
Derrick
Attachment:
nsdecimalnumber_paranoia.mm.gz
Description: GNU Zip compressed data
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden