Re: real comparison problems [Re: ASTest redux]
Re: real comparison problems [Re: ASTest redux]
- Subject: Re: real comparison problems [Re: ASTest redux]
- From: has <email@hidden>
- Date: Sat, 28 Sep 2002 15:54:35 +0100
bill fancher wrote:
>
Any time you display anything less than full precision, you get
>
"misleading" displays of the sort you mention. But even if you display
>
full precision, or don't display floating point representations at all,
>
that doesn't make the fundamental rounding problem go away, or prevent
>
"puzzling" displays:
I accept the rounding problem will always exist, and maybe I'm missing
something, but I can't understand: if AS _displays_ the result of 0.7*0.7
as 0.49, and _displays_ a literal as 0.49, it then decides to treat those
two values differently when performing a comparison. It this bit that seems
_inconsistent_ - if it treats real numbers one way in one situation, why
does it treat them differently in another?
i.e. Why not apply the same rounding processes used to display 0.49 rather
than 0.48999999999 on screen when it performs the [off-screen] comparison
operation too?
Sorry if I'm being dense, but as you can see AS's peculiar(?),
inconsistent(?) behaviour has got me totally flummoxed (again)... :p
Many thanks,
has
--
http://www.barple.pwp.blueyonder.co.uk -- The Little Page of AppleScripts
_______________________________________________
applescript-users mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/applescript-users
Do not post admin requests to the list. They will be ignored.