Re: 1.0 divided by 10 = 0.10000000000000001?
Re: 1.0 divided by 10 = 0.10000000000000001?
- Subject: Re: 1.0 divided by 10 = 0.10000000000000001?
- From: Marcel Weiher <email@hidden>
- Date: Mon, 27 Mar 2006 23:53:23 +0100
On 27 Mar 2006, at 23:24, Derrick Bass wrote:
On Mar 25, 2006, at 1:03 AM, Axman wrote:
from what i can figure out, i'm not doing anything wrong, but when i
debug it, AXdecimal is "0.10000000000000001", whis is not 1
divided by
10 is it? the result is that i end up with
123.5599975585938 when it should be 123.56. any ideas? Thanks in
Others have mentioned that this is due to the finite precision of
floating point types. Also important is that floats and doubles are
internally represented in binary rather than decimal and 0.1 cannot
be exactly represented as a finite binary digit string. Just like
2/3 cannot be represented as a finite length decimal sequence.
For more than you ever wanted to know, see the excellent article in
the link that Nick Zitmann provided. Then investigate
NSDecimalNumber if exact base-10 arithmetic is important in your
application.
Begin forwarded message:
From: Chris Kane <email@hidden>
Date: 23 August 2005 17:27:29 BDT
To: Cocoa Mailing List <email@hidden>
Subject: Re: Float behaviour
On Aug 22, 2005, at 6:40 PM, David Phillip Oster wrote:
At 11:24 AM -0700 8/22/05, ObjM2 <email@hidden>
You'll need a BCD library (BCD = binary coded decimals => no
rounding
errors)
It shouldn't be all too difficult to write a Cocoa class for BCDs
though.
Already in Cocoa: NSDecimalNumber
Actually no. NSDecimalNumber is a floating point representation,
with mantissa and exponent, just like doubles. It's just able to
represent a larger range/more precision than what had been
available natively from C-based languages and the compiler. "long
double" -- available in gcc 4.0 -- is about the same precision and
a much greater value range. Not to mention that any "native"
compiler support is going to be a thousand times faster than doing
arithmetic with NSDecimalNumber.
Chris Kane
Cocoa Framework, Apple
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden