Re: Yosemite decimal number conversion error
Re: Yosemite decimal number conversion error
- Subject: Re: Yosemite decimal number conversion error
- From: Shane Stanley <email@hidden>
- Date: Wed, 29 Oct 2014 17:03:54 +1100
On 29 Oct 2014, at 3:22 pm, Takaaki Naganoya <email@hidden> wrote:
>
> --> {-100, -1, 0.10000000149, 1, 1.100000023842, 10}--Result (!!!!)
The problem is that many values can't be accurately stored as binary values -- there is always some rounding involved. The value 0.1 in binary is:
1100110011001100110011001100110011... ad infinitum.
So somewhere along the line, some rounding is done. The net result is that the value depends on the precision it's stored in.
But apart from looking a bit odd, what practical problem is there with the result you have above?
--
Shane Stanley <email@hidden>
<www.macosxautomation.com/applescript/apps/>
_______________________________________________
Do not post admin requests to the list. They will be ignored.
AppleScript-Users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
Archives: http://lists.apple.com/archives/applescript-users
This email sent to email@hidden