Re: number formatting
Re: number formatting
- Subject: Re: number formatting
- From: Jim Correia <email@hidden>
- Date: Sat, 23 Feb 2002 12:40:13 -0500
On Saturday, February 23, 2002, at 12:16 PM, Lance Bland wrote:
Sorry to belabor the subject, but let me put it another way ...
Say we've designed our own interface building application (hint) and we
have a text field. Now, I have to figure out the default behavior of a
text field that will satisfy most users. (I don't want a default behavior
that will satisfy almost no users or purpose). Do I design the default
behavior for the computer (i.e.: 0.4 shows up as .39999 or whatever
truncation effect it is) or do I design it for a human (who would most
likely want 0.4 to show up as 0.4)?
Lance, have you read what people have been telling you? Floating point
values don't have exact representations inside the machine.
Run this code and watch it in the debugger.
float value = 0.35;
printf("floating point value: %f\n", value);
The value is represented in the machine as 0.34999...
To force it to show up as 0.35 in a text field the text field would have
to force the precision to 2 places. Then you'd be still complaining about
the cases where you wanted more or less precision.
_______________________________________________
cocoa-dev mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/cocoa-dev
Do not post admin requests to the list. They will be ignored.