Re: more float rounding info
Re: more float rounding info
- Subject: Re: more float rounding info
- From: "Jay Rimalrick" <email@hidden>
- Date: Sun, 30 Nov 2003 01:39:46 -0500
thanks to everybody who has helped with this problem. I now understand why
the error occurs but is there a simple way to get around this rounding error?
Thanks in advance
---------- Original Message ----------------------------------
From: Don Yacktman <email@hidden>
Date: Sat, 29 Nov 2003 23:50:36 -0600
>
>
On Saturday, November 29, 2003, at 10:32 PM, Jay Rimalrick wrote:
>
>
> here is my problem upfront and personal:
>
>
>
> float aFloat = 0.615;
>
>
>
> int aInt = 0;
>
>
>
> aFloat = aFloat * 100.0;
>
>
>
> aFloat = round(aFloat); //roundf acts the same way as round for me
>
>
>
> aInt = (int)aFloat; //just making sure the decimal values are gone
>
>
>
> aFloat = aInt / 100.0;
>
>
>
> NSLog(@"%f", aInt / 100.0); //this outputs 0.62
>
>
>
> NSLog(@"%f", aFloat); //this outputs 0.619999
>
>
>
>
>
>
>
> -this is very weird to me, any and all help is greatly appreciated
>
>
Spend some time reading about floating point numbers and it will start
>
to make sense. Basically you're getting a rounding error due to the
>
division and the conversion from decimal to binary. For a roughly
>
analogous example that might help you towards understanding, consider
>
how 1/3 cannot be fully represented in decimal. 0.333333 is close, but
>
with a finite number of digits, can never be exactly 1/3. Multiply by
>
three and you get 0.999999 instead of 1.0. There's always a minor
>
error there.
>
>
Now your problem is similar, but not quite the same. In binary, the
>
number 1/2 (or 0.5) can be represented exactly, as can 1/4 (0.25), 1/8
>
(0.125), 1/16 (0.0625) and additive combinations of those. But the
>
number 1/10, when converted to base two, is infinitely repeating.
>
Which means that, like 1/3 in base 10, there will be an error
>
introduced by the division.
>
>
Read a reference on the printf function; formats that limit the number
>
of digits after the decimal might help you somewhat. (I think that
>
%0.2f might be about what you want.)
>
>
--
>
Later,
>
>
Don Yacktman
>
email@hidden
>
>
________________________________________________________________
Sent via the WebMail system at 1st.net
_______________________________________________
cocoa-dev mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/cocoa-dev
Do not post admin requests to the list. They will be ignored.