Re: inconsistent float behaviour
Re: inconsistent float behaviour
- Subject: Re: inconsistent float behaviour
- From: "Kyle Sluder" <email@hidden>
- Date: Fri, 10 Oct 2008 21:58:57 -0400
On Fri, Oct 10, 2008 at 9:33 PM, Steven Hamilton <email@hidden> wrote:
> I'm still unsure why I'm getting 4 decimal places as I know for fact that
> the data is 2 decimal places. However, it seems my approach is wrong. I'll
> convert to int's and NSDecimalNumbers instead and remove all floats.
The problem is that you don't really know that the data has two
decimal places. As Mike Ash pointed out, IEEE floating point can
rarely exactly specify a number that only has two decimal places. So
though you might write a literal "1.23" in your code, the compiler
converts it to a number like "1.2300000000000001". In fact, go ahead
and launch Terminal.app and run `python`. Type 1.23, and the
interpreter will give you back "1.23". Then type "1.23 * 10", and
watch as the interpreter gives you "12.300000000000001".
This article, "7 Top Tips for Coding with Currency", just made it to
progamming.reddit.com about four days ago. It gives a very nice
summary of the issues involved when dealing with currency -- not
surprisingly, the first one is "work with integer minor units, and
never ever ever use floating point numbers".
--Kyle Sluder
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden