Re: Accurate decimal numbers?
Re: Accurate decimal numbers?
- Subject: Re: Accurate decimal numbers?
- From: mark <email@hidden>
- Date: Fri, 31 Dec 2004 17:32:57 -0800
>> I'd like to convert decimal strings to unsigned longs; I'm not sure
>> the best Cocoa way. I know that scanFloat, scanDouble doesn't work,
>> because of conversion errors (i.e., "1.55"-->1.54999). In my case,
>> its very important to convert the string accurately, as I'd be
>> converting something like 1.555m --> 1555mm.
>
> Just to be clear: scanFloat and scanDouble are not the issue, they're
> doing their job correctly. The fact that you're forced to represent a
> fraction in binary is the problem. In other words, the value you get
> from scanFloat is correct in the sense that you cannot tell the
> difference between 1.55 and 1.549999999..., they are the same number.
> (Just like 1.0 and 0.99999... are the *same* number.)
>
> Now, in your case, what you really want to do is keep track of the
> significant digits. I'd suggest something like:
>
> typedef struct {
> double value;
> uint32_t significantDigits;
> } PreciseValue;
> (How to actually figure out how many values to the right of the decimal
> point there are is an exercise for the reader. In other words, I
> really don't feel like coding it right now; I'm going to go celebrate
> 2005 instead.)
>
> Then you know exactly how many digits your final answer will have. You
> can then do something like printf("%.[sigDigitsHere]f", x.value); which
> will do what you want.
In my case, I need to convert the value to a base (long integer) unit (i.e.,
1.5m has to be converted to 1500 mm, if "mm" is the base unit). The issue
isn't displaying, but converting to the base unit. The user would notice if
they type in 1.5m and I convert it to 1499mm as that wasn't what they typed
in.
Maybe I just should go back to scanDouble and add in an estimation bit at
the limit of my accuracy, to force the rounding that I want (just using the
rint() that you had mentioned in the 2nd part)
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden