Try to write 1.555 using the sum of only powers of two, and you'll see
the issue (it cannot be done in a finite number of digits). In binary,
1.555 would be approx: 1.1000111000010100011...(continues on for
infinity)
This is the same reason that one cannot write the value of 1/3 in a
finite number of decimal digits.
On Fri, 31 Dec 2004 13:18:23 -0800, Mark Dawson <email@hidden> wrote:
> doing a "[scanner scanDouble:&val] on the string "1.555" sets the "val"
> to 1.554999999…, "1.55" to 1.55.
>
> scanFloat on "1.55" set its to 1.54999995, "1.555" to 1.5549999995
>
> While I know there are precision issues with floats and doubles, I
> can't understand why a "1.55" number can't be converted "cleanly". It
> seems like these numbers should be way within the precision boundaries
> of a float precision. Am I missing something?
>
> Thanks,
>
> Mark
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Cocoa-dev mailing list (email@hidden)
> Help/Unsubscribe/Update your Subscription:
> http://lists.apple.com/mailman/options/cocoa-dev/email@hidden
>
> This email sent to email@hidden
>
--
Clark S. Cox III
email@hidden
http://www.livejournal.com/users/clarkcox3/http://homepage.mac.com/clarkcox3/
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/email@hidden
This email sent to email@hidden
