Re: inconsistent float behaviour
Re: inconsistent float behaviour
- Subject: Re: inconsistent float behaviour
- From: Dave Carrigan <email@hidden>
- Date: Fri, 10 Oct 2008 09:54:43 -0700
On Oct 10, 2008, at 4:04 AM, Steven Hamilton wrote:
Hi folks,
I think I need some help understanding floats.
I have a class that calculates a balance and places it in a table
column. At the end of this method I NSLog the output like so;
NSLog(@"current balance is: %@",balance);
and get
current balance is: 18976.69
in the console. So far so good.
The method plonks the balance into a table and returns it to whence
it came. The table is sourced by the normal datasource delegate
methods and the column has a number formatter on it set to Decimal
#,##0,00## with half even rounding. I placed the last ##'s in there
to check that the floats weren't producing decimals with more than 2
places but in my table column I get 18976.6995 which doesn't make
sense. The float calculation is exactly 2 decimal places, I know
that for a fact and NSLog proves it. Yet somehow, the table
datasource appears to produce random decimal digits where i have the
last ##.
Am I totally misunderstanding something about floats? Or is the
extra ##'s in the formatter making things up?
There are an infinite number of floating point values between any two
integer numbers, so it is not possible to represent every single
possible float value using a fixed-width binary number. You're most
likely seeing a number that can't be exactly represented.
If you're dealing with currency, you should probably be using integer
arithmetic, and store all values as pennies or tenths of pennies,
depending on the precision you need.
--
Dave Carrigan
email@hidden
Seattle, WA, USA
Attachment:
PGP.sig
Description: This is a digitally signed message part
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden