Re: number with one decimal
Re: number with one decimal
- Subject: Re: number with one decimal
- From: Bill Briggs <email@hidden>
- Date: Mon, 18 Jul 2011 22:00:40 -0300
At 9:26 PM +0200 7/18/11, Willem wrote:
>I am stuck with rounding an number to one decimal, something strange is happening. I have this applescript in Xcode 3.1.4:
>
>line 1: set aantalvragen to contents of text field "aantalvragenveld" as integer -- aantalvragenveld is a field where i can put in the number of questions on a test (voor my students)
>line 2: set aantalgoed to contents of text field "aantalgoedveld" as integer --aantalgoedveld is a field where i can put the number of correct questions
>line 3: set resultaat to (aantalgoed * 90 / aantalvragen) + 10 as integer --now the result is between 10 and 100, but it schould be between 1 and 10 (in Holland we like this ;)
>line 4: set contents of text field "Resultaatveld" to resultaat / 10 --and now its between 1 and 10 with one decimal.
>
>But, the strange thing is when aantalvragen is 10 and aantalgoed is 8 , then in line 3 resultaat is 82 (thats oke), but in line 4 its 8,19999999999999 (this should be 8,2)
>
>So applescript thinks that 82 / 10 = 8,1999999999
>
>It does not work to round things.
>
>As i said, i am stuck
>
>hope you can help
This problem arises, as others have obliquely eluded to, because the digital computer is a "state machine" and the digital representation of real numbers on the number line is patchy (and this is not at all related to BCD encoding, which will manifest entirely different problems).
The more bits in the representation (register width), the more numbers on the number line you can represent, but there will always be numbers that can't be accurately represented in binary, no matter how many bits wide the floating point register is. There are only 2^n states that can be represented (with exponents taking up some number of bits in the register), and the number line is infinitely densely packed. You get unavoidable holes, so odd things can show up. This is why CS/CMPE/EE students study numerical methods. They need to learn of the methods, but also of the limits of the machine when doing calculations in which errors can accumulate.
The truth is that we do rather well, but there are times when the representation causes things to fall apart. As you are experiencing now.
I like the solution that was suggested using bc, because bc, if you look at the rather short (about three pages of) source code, doesn't rely on the way the machine represents numbers internally in binary. It does arithmetic using base 10 digits, using the same algorithms you and I do with a pencil and paper. It's actually very elegant, and the code is very fast. And for things like cryptography, where precise integers, not approximate reals, are needed, it's the only way to go.
There are clearly ways to get AS to cough up the right results, after all, HP calculators were avoiding this kind of issue back in the early 70s and they weren't using a "bc" like approach, but bc is a very cool tool. You can even write your own function library that loads on launch and do all manner of very interesting things. If you crunch a lot of numbers on your Mac, you owe it to yourself to get familiar with "bc". I have numerous AppleScripts that toss things to bc for processing and the results are instant. For those seeking AppleScript purity, the other methods suggested will get you there.
- web
_______________________________________________
Do not post admin requests to the list. They will be ignored.
AppleScript-Users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
Archives: http://lists.apple.com/archives/applescript-users
This email sent to email@hidden