• Open Menu Close Menu
  • Apple
  • Shopping Bag
  • Apple
  • Mac
  • iPad
  • iPhone
  • Watch
  • TV
  • Music
  • Support
  • Search apple.com
  • Shopping Bag

Lists

Open Menu Close Menu
  • Terms and Conditions
  • Lists hosted on this site
  • Email the Postmaster
  • Tips for posting to public mailing lists
Re: Same code gives different answers in 2 different scripts
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Same code gives different answers in 2 different scripts


  • Subject: Re: Same code gives different answers in 2 different scripts
  • From: Rob Lewis <email@hidden>
  • Date: Tue, 2 Jun 2009 15:19:05 -0700


Thanks, Chris. I get it about floating point rounding errors, but what I found mystifying was that the EXACT SAME CODE gave different answers (one right, one wrong) when run in two different scripts (on the same Mac). Can't be cosmic rays because it's repeatable. 

Do you have a better rounding algorithm that will fix this? 


This might be my best AS stumper yet. I have a situation where the

IDENTICAL code gives different answers in two different scripts.


Here's the code:


property form_height : ((1.75 - 0.7) * 72) --=75.6


to findRound of num given numDecimals:n

set x to 10 ^ n

return (((num * x) + 0.5) div 1) / x

end findRound


set rounded_height to findRound of form_height given numDecimals:2



When run in a Script Editor window by itself, this gives the expected

result of 75.6 (apparently trailing zeroes are trimmed).


This very same code, when incorporated in a larger script, gives the  

result

75.59999999999999. None of its parameters are touched by any other  

part of

the program.


The program gets the right answer when rounding other numbers.


Why won't it round a simple number? Why does it give a wrong answer?


Long answer: <http://docs.sun.com/source/806-3568/ncg_goldberg.html>.


Short answer: Floating-point math such as AppleScript uses is not  

exact.  What Script Editor displays as the result is often not the  

precise value of the number, and operations such as "div 1" can reveal  

the tiny discrepancies.  AppleScript is not unique in this; most  

programming languages use the same floating-point standard.



--Chris Nebel

AppleScript Engineering


 _______________________________________________
Do not post admin requests to the list. They will be ignored.
AppleScript-Users mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:
Archives: http://lists.apple.com/archives/applescript-users

This email sent to email@hidden

  • Follow-Ups:
    • Re: Same code gives different answers in 2 different scripts
      • From: Chris Page <email@hidden>
    • Re: Same code gives different answers in 2 different scripts
      • From: Paul Scott <email@hidden>
  • Prev by Date: Re: AppleScript Studio Application - Annoying Delays
  • Next by Date: Re: Same code gives different answers in 2 different scripts
  • Previous by thread: Re: Same code gives different answers in 2 different scripts
  • Next by thread: Re: Same code gives different answers in 2 different scripts
  • Index(es):
    • Date
    • Thread