Re: NSDecimal chokes on very small numbers
Re: NSDecimal chokes on very small numbers
- Subject: Re: NSDecimal chokes on very small numbers
- From: Tom Bernard <email@hidden>
- Date: Tue, 08 Dec 2009 03:26:27 -0700
- Thread-topic: NSDecimal chokes on very small numbers
My work with NSDecimal values has turned up several bugs and a documentation
error. Most of these bugs are extant under Mac OS X 10.5.7, and are FIXED
under Mac OS X 10.6 (10A380).
For the record:
All bugs are extant under Mac OS X 10.5.7
Extant under Mac OS X 10.6 (10A380):
NSDecimalPower() chokes on small numbers
NSDecimalIsNotANumber() incorrect documentation
FIXED under Mac OS X 10.6 (10A380):
NSDecimalAdd() bug
NSDecimalSubtract() bug
NSDecimalCompare() bug
NSDecimalNormalize() drops very small fraction
NSDecimalAdd() and NSDecimalSubtract() return incorrect results when adding
or subtracting a very small number to or from a larger number.
NSDecimalCompare() incorrectly compared NSDecimal one with NSDecimal
1+1e-34, returning NSOrderedSame when NSOrderedAscending was expected.
NSDecimalNormalize() normalized 1+1e-34 to 1.0 .
These issues are extant under Mac OS X 10.5.7 and FIXED under Mac OS X 10.6
(10A380)
Not fixed is the NSDecimalPower() bug already noted. Also, the documentation
for NSDecimalIsNotANumber() has the return values YES and NO reversed.
Bug reports:
NSDecimalPower(): 7452166
NSDecimalIsNotANumber() documentation: 7452179
Tom Bernard
email@hidden
on 11/21/09 7:41 AM, Tom Bernard wrote:
> Before reporting this as a bug to Apple's Bug Reporter, I would like
> feedback from the community. I am working in Leopard. Has this been fixed in
> Snow Leopard? Is there something else I am overlooking?
>
> NSDecimalPower returns an unexpected result when I raise 1e-35 to the 4th
> power. I understand that the result is too small to be supported by
> NSDecimal. The returned result:
>
> 100000000000000000000000000000000000000000000000000000000000000000000000000\
> 000000000000000000000000000000000000000000 and err = NSCalculationNoError
>
> does not make sense to me. I was expecting a return of 0 and an err =
> NSCalculationLossOfPrecision.
>
>
> To reproduce:
>
> 1) create a new Foundation Tool and implement the following code:
>
> #import <Foundation/Foundation.h>
>
> #define NSDStr(x) NSDecimalString(&x, nil)
>
>
> int main (int argc, const char * argv[]) {
> NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
>
> NSCalculationError err;
>
> // NSDecimalPower bug
> // pow(0.0000000000000000000000000000000000001, 4);
> // result is less than minimum NSDecimal; NSDecimalPower should return 0
> with err = NSCalculationLossOfPrecision
> // (NSCalculationOverflow, NSCalculationUnderflow also understandable)
> //
> // instead returns
> 1000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000
> // with err = NSCalculationNoError
>
> NSDecimal oneEminusThirtyFive, oneEminusThirtyFiveToFourthPower;
> NSString *oneEminusThirtyFiveStr,
> *oneEminusThirtyFiveToFourthPowerStr;
> NSUInteger n;
>
> n = 4;
> oneEminusThirtyFive = [[NSDecimalNumber decimalNumberWithMantissa:1
> exponent:-35 isNegative:NO] decimalValue];
> err =
> NSDecimalPower(&oneEminusThirtyFiveToFourthPower, &oneEminusThirtyFive, n,
> NSRoundPlain);
>
> oneEminusThirtyFiveStr = NSDStr(oneEminusThirtyFive);
> oneEminusThirtyFiveToFourthPowerStr =
> NSDStr(oneEminusThirtyFiveToFourthPower);
>
> printf("oneEminusThirtyFive = %s\n",
> [oneEminusThirtyFiveStr UTF8String]);
> printf("oneEminusThirtyFiveToFourthPower = %s\n",
> [oneEminusThirtyFiveToFourthPowerStr UTF8String]);
> printf("err = %u\n", err);
> printf("\n");
>
>
> [pool drain];
> return 0;
> }
>
>
> The above code outputs:
>
> oneEminusThirtyFive = 0.00000000000000000000000000000000001
> oneEminusThirtyFiveToFourthPower =
> 1000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000
> err = 0
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden