doing a "[scanner scanDouble:&val] on the string "1.555" sets the "val"
to 1.554999999…, "1.55" to 1.55.
scanFloat on "1.55" set its to 1.54999995, "1.555" to 1.5549999995
While I know there are precision issues with floats and doubles, I
can't understand why a "1.55" number can't be converted "cleanly". It
seems like these numbers should be way within the precision boundaries
of a float precision. Am I missing something?
Thanks,
Mark
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/email@hidden
