• Open Menu Close Menu
  • Apple
  • Shopping Bag
  • Apple
  • Mac
  • iPad
  • iPhone
  • Watch
  • TV
  • Music
  • Support
  • Search apple.com
  • Shopping Bag

Lists

Open Menu Close Menu
  • Terms and Conditions
  • Lists hosted on this site
  • Email the Postmaster
  • Tips for posting to public mailing lists
Re: number with one decimal
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: number with one decimal


  • Subject: Re: number with one decimal
  • From: "Mark J. Reed" <email@hidden>
  • Date: Mon, 18 Jul 2011 19:35:16 -0400

On Mon, Jul 18, 2011 at 7:28 PM, Steve Thompson <email@hidden> wrote:

On 18 Jul 2011, at 23:16, Mark J. Reed wrote:

Any computer software using floating-point arithmetic shares
AppleScript's opinion on that.  

Not true. Some software handles it better than others.

No floating-point implementation can exactly represent 0.1.  It may "handle" that fact better, somehow, but it can only work around it, not avoid it completely.
 
This is a rounding problem, but if all
you care about is the output, just fake the division when you do the
string conversion yourself.  Assuming scale is an integer:


Scale is always an integer. You can't have 3 and a half decimal places

Wrong variable.  I was assuming scale was the value to be rounded.  My point is, if you have a number that you want to divide by 10 for output,  you can fake it by inserting a decimal separator into the string at the appropriate place instead of actually doing the division.

I think you misread my code

Sorry, I just didn't pay attention to the details of which variable was named what.  Yes, you can use bc - or dc, or other programs/script languages that similarly support arbitrary-precision decimal arithmetic.  That's a fine general solution;  I was offering a simple pure-AS solution for the OP's specific goal.



--
Mark J. Reed <email@hidden>

 _______________________________________________
Do not post admin requests to the list. They will be ignored.
AppleScript-Users mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:
Archives: http://lists.apple.com/archives/applescript-users

This email sent to email@hidden

References: 
 >number with one decimal (From: Willem <email@hidden>)
 >Re: number with one decimal (From: Steve Thompson <email@hidden>)
 >Re: number with one decimal (From: "Mark J. Reed" <email@hidden>)
 >Re: number with one decimal (From: Steve Thompson <email@hidden>)

  • Prev by Date: Re: number with one decimal
  • Next by Date: Re: number with one decimal
  • Previous by thread: Re: number with one decimal
  • Next by thread: Re: number with one decimal
  • Index(es):
    • Date
    • Thread