Re: Runded Beyond Recognition
Re: Runded Beyond Recognition
- Subject: Re: Runded Beyond Recognition
- From: Doug McNutt <email@hidden>
- Date: Sat, 21 Feb 2004 18:40:56 -0700
At 18:27 -0600 2/21/04, ehsan saffari wrote:
>
I have a number result like this: 2.45305722033565E+6
>
I need to display it in non scientific form: 2453057.22033565, either as
>
a string or real.
>
SNIP
>
Max number of decimal places in my number results will be 8.
The number of decimal places, meaning the number of digits after the decimal point, is NOT the controlling consideration. The number of interest in the number of significant digits, whether they appear before or after the decimal point. IEEE doubles, which are the basic numerical construct of Excel are limited to 42 bits of precision. That translates to 42 * log(10)2 or about 15 digits. You are not ever going to to better than that in Excel.
The price you pay for 10 ^ +/- 308 in range is that you simply cannot do arithmetic with precision better than 15 significant digits. (And that includes the digits before the point.)
If you want more precision than that go to Waterloo Maple or Mathematica. Excel simply won't do that kind of arithmetic. The IBM 1620 - ancient - could that kind of arithmetic because it expressed numbers in 4 bit decimal integers up the the memory capacity of the machine. It died long ago.
And, incidentally, Excel peecee and other 32 bit machines already use 64 or 80 bit arithmetic internally. Updating to a 64 bit microprocessor might make things faster but it will not improve the precision of the results as stored in memory.
--
--> There are 10 kinds of people: those who understand binary, and those who don't <--
_______________________________________________
applescript-users mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/applescript-users
Do not post admin requests to the list. They will be ignored.