• Open Menu Close Menu
  • Apple
  • Shopping Bag
  • Apple
  • Mac
  • iPad
  • iPhone
  • Watch
  • TV
  • Music
  • Support
  • Search apple.com
  • Shopping Bag

Lists

Open Menu Close Menu
  • Terms and Conditions
  • Lists hosted on this site
  • Email the Postmaster
  • Tips for posting to public mailing lists
Re: Same code gives different answers in 2 different scripts
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Same code gives different answers in 2 different scripts


  • Subject: Re: Same code gives different answers in 2 different scripts
  • From: Paul Scott <email@hidden>
  • Date: Tue, 2 Jun 2009 15:38:52 -0700

Maybe I'm missing it, but I don't see how  the example you gave should have a rounding error as there is sufficient  resolution in the mantissa.

Paul

On Jun 2, 2009, at 3:19 PM, Rob Lewis wrote:


Thanks, Chris. I get it about floating point rounding errors, but what I found mystifying was that the EXACT SAME CODE gave different answers (one right, one wrong) when run in two different scripts (on the same Mac). Can't be cosmic rays because it's repeatable. 

Do you have a better rounding algorithm that will fix this? 


This might be my best AS stumper yet. I have a situation where the
IDENTICAL code gives different answers in two different scripts.

Here's the code:

property form_height : ((1.75 - 0.7) * 72) --=75.6

to findRound of num given numDecimals:n
set x to 10 ^ n
return (((num * x) + 0.5) div 1) / x
end findRound

set rounded_height to findRound of form_height given numDecimals:2


When run in a Script Editor window by itself, this gives the expected
result of 75.6 (apparently trailing zeroes are trimmed).

This very same code, when incorporated in a larger script, gives the  
result
75.59999999999999. None of its parameters are touched by any other  
part of
the program.

The program gets the right answer when rounding other numbers.

Why won't it round a simple number? Why does it give a wrong answer?

Long answer: <http://docs.sun.com/source/806-3568/ncg_goldberg.html>.

Short answer: Floating-point math such as AppleScript uses is not  
exact.  What Script Editor displays as the result is often not the  
precise value of the number, and operations such as "div 1" can reveal  
the tiny discrepancies.  AppleScript is not unique in this; most  
programming languages use the same floating-point standard.


--Chris Nebel
AppleScript Engineering




 _______________________________________________
Do not post admin requests to the list. They will be ignored.
AppleScript-Users mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:
Archives: http://lists.apple.com/archives/applescript-users

This email sent to email@hidden

References: 
 >Re: Same code gives different answers in 2 different scripts (From: Rob Lewis <email@hidden>)

  • Prev by Date: Re: Same code gives different answers in 2 different scripts
  • Next by Date: Re: Same code gives different answers in 2 different scripts
  • Previous by thread: Re: Same code gives different answers in 2 different scripts
  • Next by thread: Re: Same code gives different answers in 2 different scripts
  • Index(es):
    • Date
    • Thread