Re: number formatting
Re: number formatting
- Subject: Re: number formatting
- From: Jim Correia <email@hidden>
- Date: Sun, 3 Mar 2002 20:55:10 -0500
On Sunday, March 3, 2002, at 03:48 AM, Michael Gersten wrote:
Do I design the default
behavior for the computer (i.e.: 0.4 shows up as .39999 or whatever
truncation effect it is) or do I design it for a human (who would most
likely want 0.4 to show up as 0.4)?
Lance, have you read what people have been telling you? Floating point
values don't have exact representations inside the machine.
Run this code and watch it in the debugger.
float value = 0.35;
printf("floating point value: %f\n", value);
The value is represented in the machine as 0.34999...
To force it to show up as 0.35 in a text field the text field would have
to force the precision to 2 places. Then you'd be still complaining about
the cases where you wanted more or less precision.
I must disagree with this.
.34999999 is a truncated, not rounded, value. It is the truncation of
.349999999 (notice one more digit on this line).
If you ROUND instead of truncating, then you get .35
You miss the point. If the framework ROUNDS to 2 places, it will be
appropriate for you, but not for someone else. There is no way to look at
the number and empirically determine the number of significant digits the
programmer/user wants displayed. ROUNDing to any number of places doesn't
solve the problem, it just pushes the problem around a bit, doesn't solve
it. To solve it, use a formatter.
_______________________________________________
cocoa-dev mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/cocoa-dev
Do not post admin requests to the list. They will be ignored.