Re: number formatting
Re: number formatting
- Subject: Re: number formatting
- From: Graeme Hiebert <email@hidden>
- Date: Mon, 11 Mar 2002 23:14:49 -0800
On Monday, March 11, 2002, at 11:00 PM, Michael Gersten wrote:
<snip>
Once you've gone through all the digits available -- roughly 6 or 7
for single, 14 or 15 for double -- then going any farther just
generates significant roundoff errors, like pentium math bugs. Stop at
that point, round off the last digit that was generated, and you've
got the final answer.
You're right, of course. I just get all excited about the math. (I
don't often get a chance to exercise that part of my brain. :^)
Put it another way: which is the better value for 2/3:
.666666666666666666660000000000000000000000
.666666666666666666670000000000000000000000
Those that say "truncate" will go for the first.
Those that say "round at the last digit that has source data" (me)
will go for the second.
<snip>
Again: the current default is to truncate when running out of digits;
it makes more sense to round that last digit when you run out instead
of truncating.
I myself don't give a flying fling-flang whether it truncates or
rounds. What is more important, IMHO, is that my program will give the
same display a year or two from now as it does today. Since it has
truncated instead of rounding for many years now, it would be a bad idea
to change it.
It's really pointless worrying about what is right or wrong. It works
the way it works, and it should continue to do so. It is deadly simple
to apply a formatter to a text field, so the point is rather moot anyway.
-g
_______________________________________________
cocoa-dev mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/cocoa-dev
Do not post admin requests to the list. They will be ignored.