Re: double precision in x-code
Re: double precision in x-code
- Subject: Re: double precision in x-code
- From: Lennart Thelander <email@hidden>
- Date: Sun, 26 Sep 2010 12:05:48 +0200
- Thread-topic: double precision in x-code
Hello,
To simplify, a fractional value is stored by adding (1 or 0 multiplied with)
1/2, 1/4, 1/8, 1/16, 1/32 and so on. So 0.3 can't be exactly represented.
The closest value you get is 0.29999999999999993.
So never test for equal when it comes to floating point calculations. Do
something like
if ( abs(myVar - 0.3) < 0.000000001 )
Regards
Lennart Thelander
On 10-09-26 10.57, "Juan Vesa" <email@hidden> wrote:
> hello,
> when i inspect the values of double variables in the x-code debugger
> i note they are not precise, for example i might set a variable
> double tmp = 0.3, and in the debugegr inspector i will see the value
> 0.29999999999999993. i'm afraid this error may be accumulating and
> not making some calculations as precise as possible. is this
> standard? is the actual value in memory 0.3 or is the debugger
> window just showing something wonky?
> any advice would be greatly appreciated,
> cheers,
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Xcode-users mailing list (email@hidden)
> Help/Unsubscribe/Update your Subscription:
>
m
>
> This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Xcode-users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden