Lists

Terms and Conditions
Lists hosted on this site
Email the Postmaster
Tips for posting to public mailing lists

Re: 1.0 divided by 10 = 0.10000000000000001?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 1.0 divided by 10 = 0.10000000000000001?

Subject: Re: 1.0 divided by 10 = 0.10000000000000001?
From: Ricky Sharp <email@hidden>
Date: Mon, 27 Mar 2006 15:57:41 -0600


On Mar 25, 2006, at 1:03 AM, Axman wrote:

from what i can figure out, i'm not doing anything wrong, but when i debug it, AXdecimal is "0.10000000000000001", whis is not 1 divided by 10 is it? the result is that i end up with 123.5599975585938 when it should be 123.56. any ideas?

This is because floating point values, while accurate, can only be so precise.

Look up topics on "Numerical Analysis" and how numbers are stored in floating point form.


___________________________________________________________
Ricky A. Sharp         mailto:email@hidden
Instant Interactive(tm)   http://www.instantinteractive.com

_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden



References:  
  >1.0 divided by 10 = 0.10000000000000001? (From: Axman <email@hidden>)




Prev by Date:
Re: Memory Management question

Next by Date:
Re: 1.0 divided by 10 = 0.10000000000000001?

Previous by thread:
1.0 divided by 10 = 0.10000000000000001?

Next by thread:
Re: 1.0 divided by 10 = 0.10000000000000001?

Index(es):

Date
Thread