Re: why are floats flakey?
Re: why are floats flakey?
- Subject: Re: why are floats flakey?
- From: Shaun Wexler <email@hidden>
- Date: Tue, 25 May 2004 07:53:34 -0700
On May 24, 2004, at 7:23 PM, justin webster wrote:
>
why are floats inaccurate?
>
when can I use them with confidence and what can I use in their place
>
when I need absolute accuracy?
>
>
i'm trying to create a float with a value of 29.97 and I get 29.9699999
>
like this:
>
float myFloat = [myTextField floatValue]
>
where the text field contains the string "29.97df"
You'll probably have more problems than the "accuracy" of a float.
Standard 29.97 drop-frame loses two frames per day anyway! ;)~ The
correct value should be 29.97002616, which can be fully represented by
a double-precision 64-bit float. If you store 0x41efc29d as a 32-bit
float, it would be 29.970026f. FWIW, I prefer to perform SMPTE
calculations on H:M:S:F as short int's...
--
Shaun Wexler
MacFOH
http://www.macfoh.com
[demime 0.98b removed an attachment of type application/pkcs7-signature which had a name of smime.p7s]
_______________________________________________
cocoa-dev mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/cocoa-dev
Do not post admin requests to the list. They will be ignored.