Re: how to format decimal precision of a real?
Re: how to format decimal precision of a real?
- Subject: Re: how to format decimal precision of a real?
- From: Kai Edwards <email@hidden>
- Date: Mon, 04 Mar 2002 00:00:14 +0000
on Sun, 3 Mar 2002 09:23:06 -0800, John W Baxter <email@hidden>
wrote:
>
At 23:46 -0800 3/2/2002, Bill Hoffman wrote:
>
>> The simplest answer is: 324.234234234 * 100 div 1 / 100
>
>
>
> This works fine for my needs, just what I was looking for. Thanks to all
>
> who replied.
>
>
It works for Bill's needs...but it still produces an approximation to
>
324.23 (which AppleScript then "displays properly" (or "lies about"
>
depending on one's viewpoint).
>
>
It's easy to produce a different approximation to 324.23:
>
(324.234234234 * 100 div 1 / 100) is equal to (324 + 0.2 + 0.03)
>
--> false
>
which will look the same when displayed. 324.23 can't be represented
>
exactly by the kind of floating point numbers which are being used.
>
>
Also, the suggested calculation isn't rounding, it is truncating.
Hi John!
At an early stage of this thread, I'd posted a rushed (and extremely clumsy)
suggestion to Bill's question - which gave similar results (albeit using a
more tortuous path) to the one he eventually decided to run with.
Shortly after doing this, I realised that I had wrongly implied a rounding
function and sent this correction:
----------------------------------------------------------------------
>
on 3 March 2002 6:15 am, I wrote:
>
>
> rnd(324.234234234, 2)
>
>
>
> to rnd(num, decPlaces)
>
> set e to 10 ^ decPlaces
>
> num * e div e + num * e mod e div 1 / e
>
> end rnd
>
>
On reflection, to avoid getting beaten over the head by some maths boffin who
>
knows infinitely more about these things, I suppose I should have written
>
this:
>
>
trunc(324.234234234, 2)
>
>
to trunc(num, decPlaces)
>
set e to 10 ^ decPlaces
>
num * e div e + num * e mod e div 1 / e
>
end trunc
>
>
- but I'm sure you know what I really meant, anyway! ;-)
----------------------------------------------------------------------
- Which, while it did nothing for the long-windedness of the approach, did
at least attempt to correct the impression of claimed rounding.
If the correction had appeared immediately, as intended, I would have only
suffered the embarrassment of my mathematical ineptitude.
However, as chance would have it, I was also testing another script at the
time - which I believe messed about with the default account settings in
Outlook Express. Consequently, my erratum was sent using another account -
and was 'netted' by the list's administration.
Doubtless this will surface in due course to add greater depth to my
blushes.
So, as soon as I saw your contribution, I felt that I should apologise in
advance, and assure you that my comments (when they appear) are a response
only to my own haste - and not to your perfectly accurate observations.
(snip: some excellent stuff on rounding accuracy)
>
To deal with this, Apple provides in (recent?) Standard Additions:
>
>
round: Round number to integer (defined in: StandardAdditions.osax)
>
round [real] -- the number to round
>
[rounding up/down/toward zero/to nearest/as taught in
>
school] -- the rounding direction; if omitted, rounds to nearest. "to
>
nearest" rounds .5 cases to the nearest even integer in order to decrease
>
cumulative errors. To always round .5 away from zero, use "as taught in
>
school."
>
Result: integer -- the rounded value
FWIW, I often use an older system (OS 8.1 / AS 1.1.2). - which precedes
Standard Additions.
However, the Numerics Dictionary for that configuration still contained some
rounding capabilities:
----------------------------------------------------------------------
round: Round number to integer.
round extended real -- number to round
[rounding up/down/toward zero/to nearest] -- specify rounding
direction, default is rounding to nearest
Result: integer -- rounded value
----------------------------------------------------------------------
Best wishes
Kai
--
**********************************
Kai Edwards Creative Resources
1 Compton Avenue Brighton UK
Telephone +44 (0)1273 326810
**********************************
_______________________________________________
applescript-users mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/applescript-users
Do not post admin requests to the list. They will be ignored.