Re: Same code gives different answers in 2 different scripts
Re: Same code gives different answers in 2 different scripts
- Subject: Re: Same code gives different answers in 2 different scripts
- From: Christopher Nebel <email@hidden>
- Date: Mon, 1 Jun 2009 22:23:09 -0700
On Jun 1, 2009, at 5:19 PM, rob wrote:
This might be my best AS stumper yet. I have a situation where the
IDENTICAL code gives different answers in two different scripts.
Here's the code:
property form_height : ((1.75 - 0.7) * 72) --=75.6
to findRound of num given numDecimals:n
set x to 10 ^ n
return (((num * x) + 0.5) div 1) / x
end findRound
set rounded_height to findRound of form_height given numDecimals:2
When run in a Script Editor window by itself, this gives the expected
result of 75.6 (apparently trailing zeroes are trimmed).
This very same code, when incorporated in a larger script, gives the
result
75.59999999999999. None of its parameters are touched by any other
part of
the program.
The program gets the right answer when rounding other numbers.
Why won't it round a simple number? Why does it give a wrong answer?
Long answer: <http://docs.sun.com/source/806-3568/ncg_goldberg.html>.
Short answer: Floating-point math such as AppleScript uses is not
exact. What Script Editor displays as the result is often not the
precise value of the number, and operations such as "div 1" can reveal
the tiny discrepancies. AppleScript is not unique in this; most
programming languages use the same floating-point standard.
--Chris Nebel
AppleScript Engineering
_______________________________________________
Do not post admin requests to the list. They will be ignored.
AppleScript-Users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
Archives: http://lists.apple.com/archives/applescript-users
This email sent to email@hidden