NSDecimal chokes on very small numbers
NSDecimal chokes on very small numbers
- Subject: NSDecimal chokes on very small numbers
- From: Tom Bernard <email@hidden>
- Date: Sat, 21 Nov 2009 05:25:52 -0700
- Thread-topic: NSDecimal chokes on very small numbers
Before reporting this as a bug to Apple's Bug Reporter, I would like
feedback from the community. I am working in Leopard. Has this been fixed in
Snow Leopard? Is there something else I am overlooking?
NSDecimalPower returns an unexpected result when I raise 1e-35 to the 4th
power. I understand that the result is too small to be supported by
NSDecimal. The returned result:
100000000000000000000000000000000000000000000000000000000000000000000000000\
000000000000000000000000000000000000000000 and err = NSCalculationNoError
does not make sense to me. I was expecting a return of 0 and an err =
NSCalculationLossOfPrecision.
To reproduce:
1) create a new Foundation Tool and implement the following code:
#import <Foundation/Foundation.h>
#define NSDStr(x) NSDecimalString(&x, nil)
int main (int argc, const char * argv[]) {
NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
NSCalculationError err;
// NSDecimalPower bug
// pow(0.0000000000000000000000000000000000001, 4);
// result is less than minimum NSDecimal; NSDecimalPower should return 0
with err = NSCalculationLossOfPrecision
// (NSCalculationOverflow, NSCalculationUnderflow also understandable)
//
// instead returns
1000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000
// with err = NSCalculationNoError
NSDecimal oneEminusThirtyFive, oneEminusThirtyFiveToFourthPower;
NSString *oneEminusThirtyFiveStr,
*oneEminusThirtyFiveToFourthPowerStr;
NSUInteger n;
n = 4;
oneEminusThirtyFive = [[NSDecimalNumber decimalNumberWithMantissa:1
exponent:-35 isNegative:NO] decimalValue];
err =
NSDecimalPower(&oneEminusThirtyFiveToFourthPower, &oneEminusThirtyFive, n,
NSRoundPlain);
oneEminusThirtyFiveStr = NSDStr(oneEminusThirtyFive);
oneEminusThirtyFiveToFourthPowerStr =
NSDStr(oneEminusThirtyFiveToFourthPower);
printf("oneEminusThirtyFive = %s\n",
[oneEminusThirtyFiveStr UTF8String]);
printf("oneEminusThirtyFiveToFourthPower = %s\n",
[oneEminusThirtyFiveToFourthPowerStr UTF8String]);
printf("err = %u\n", err);
printf("\n");
[pool drain];
return 0;
}
The above code outputs:
oneEminusThirtyFive = 0.00000000000000000000000000000000001
oneEminusThirtyFiveToFourthPower =
1000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000
err = 0
Tom Bernard
email@hidden
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden