Re: Accurate decimal numbers?
Re: Accurate decimal numbers?
- Subject: Re: Accurate decimal numbers?
- From: Brendan Younger <email@hidden>
- Date: Fri, 31 Dec 2004 18:42:17 -0600
On Dec 31, 2004, at 6:08 PM, Mark Dawson wrote:
I have a couple of questions concerning how to use NSDecimal (or even
if that's the correct way to go):
(1)
I'd like to convert decimal strings to unsigned longs; I'm not sure
the best Cocoa way. I know that scanFloat, scanDouble doesn't work,
because of conversion errors (i.e., "1.55"-->1.54999). In my case,
its very important to convert the string accurately, as I'd be
converting something like 1.555m --> 1555mm.
Just to be clear: scanFloat and scanDouble are not the issue, they're
doing their job correctly. The fact that you're forced to represent a
fraction in binary is the problem. In other words, the value you get
from scanFloat is correct in the sense that you cannot tell the
difference between 1.55 and 1.549999999..., they are the same number.
(Just like 1.0 and 0.99999... are the *same* number.)
Now, in your case, what you really want to do is keep track of the
significant digits. I'd suggest something like:
typedef struct {
double value;
uint32_t significantDigits;
} PreciseValue;
(How to actually figure out how many values to the right of the decimal
point there are is an exercise for the reader. In other words, I
really don't feel like coding it right now; I'm going to go celebrate
2005 instead.)
Then you know exactly how many digits your final answer will have. You
can then do something like printf("%.[sigDigitsHere]f", x.value); which
will do what you want.
It looks like NSDecimal (scanDecimal) is the way to go, but I'm not
sure the best way to use it. I know I can use
NSDecimalMultiplyByPowerOf10 to convert the NSDecimal returned from
scanDecimal. What's the best way to convert an NSDecimal to an
unsigned long? I wasn't sure if I needed to create an
NSDecimalNumber, then do a [num double] to do a forced conversion.
(2)
Also, what's the best way to determine whether a number is a whole
number? (i.e. the fractional portion is 0-->1.0). I've usually done
something along the lines of
if (x != (float)((int)x))
where x is a float or double; is there an easier way to do this using
an NSDecimal number?
Avoiding int<->float conversions is always best. I'd suggest if(0.0 ==
fabs(x - rint(x)), but I don't claim to be the expert on this.
Brendan Younger
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden