Re: double precision errors and numbers not adding correctly
Re: double precision errors and numbers not adding correctly
- Subject: Re: double precision errors and numbers not adding correctly
- From: Jonathan Feinberg <email@hidden>
- Date: Sat, 23 Mar 2002 06:48:01 -0800
At 1:50 AM -0700 3/23/02, Chad Armstrong wrote:
But another weird problem is I've even tried to have the number 65.1
be put into the display by hard coding in the number. It still
comes up with the 65.09999... number.
The problems you're having with numbers have to do with representing
decimal numbers in binary form. See the following link (which is
written from the perspective of another programming language, but are
still relevant) for an explanation
http://www.python.org/doc/current/tut/node14.html
Also, go to a terminal prompt and type
perldoc -q decimal
When I added 65.0 + 0.1, it came up with 65.1, and did a printf()
statement, then 65.1 would print out, but if I had the precision to
print out like %2.14f, then the 65.099999... answer would appear.
Yes. As you've discovered, you must choose your representation
precision manually. In a Foundation-based program, you'd likely do
that using NSString's stringWithFormat message.
when I look at the value from the drawRect function, it sees the
original initialized value. If I change the myVar value in the
drawRect function, then it increments fine, but this does not affect
the 'other' myVar. What is going on here?
Who knows? Without the relevant source code, it's impossible to say.
--
Jonathan Feinberg email@hidden Inwood, NY, NY
http://MrFeinberg.com/
_______________________________________________
cocoa-dev mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/cocoa-dev
Do not post admin requests to the list. They will be ignored.