1.0 divided by 10 = 0.10000000000000001?
1.0 divided by 10 = 0.10000000000000001?
- Subject: 1.0 divided by 10 = 0.10000000000000001?
- From: Axman <email@hidden>
- Date: Sat, 25 Mar 2006 18:03:03 +1100
(I'm really sorry, i just sent this, but forgot to change the subject.
i had to make it more meaningful. again, sorry. Al.)
Hi all, i'm a 'noob' so if what i have to say is stupid i'm really
sorry. i've only just started programming in cocoa in the last few
days, but i am almost finished a little calculator (yay :D ) but i'm
having an annoying problem when i'm trying to use decimal numbers, i'm
using a variable that is a double (i want my little calc to be pretty
accurate, well very) and after the "." button is pressed i have it so
that this variable (which started out as 1.0) is divided by 10 and
then the number that's pressed is added to the full number that's
being worked on times (*) this variable: here's a little look:
("//debug!" is where i'm trying to find out what value AXdecimal is)
double AXdecimal = 1.0;
bool isDecimal = 0; (that is changed when the "." is pressed, so
isDecimal is true)
...(buttons get pressed here, and then get calculated by the next bit)
...}else if(isDecimal == 1){ //if the AXdecimal button had been pressed
if (isNumberOne == 1){
if (numberOne == 0){
AXdecimal = AXdecimal / 10;
numberOne = button * AXdecimal;
dec = AXdecimal; //debug!
x = numberOne;
}else if(numberOne != 0) {
AXdecimal = AXdecimal / 10;
dec = AXdecimal; //debug!
numberOne = numberOne + (button * AXdecimal);
x = numberOne;
}
}else{
if (numberTwo == 0){
AXdecimal = AXdecimal / 10;
dec = AXdecimal; //debug!
numberTwo = button * AXdecimal;
x = numberTwo;
}else if(numberTwo != 0) {
AXdecimal = AXdecimal / 10;
dec = AXdecimal; //debug!
numberTwo = numberTwo + (button * AXdecimal);
x = numberTwo;
}
}
....
from what i can figure out, i'm not doing anything wrong, but when i
debug it, AXdecimal is "0.10000000000000001", whis is not 1 divided by
10 is it? the result is that i end up with
123.5599975585938 when it should be 123.56. any ideas? Thanks in
advance, and if you can help, could you mail me directly at
email@hidden as well as cocoa-dev@... (because i get the daily
round up thing, so i might get it today, and also other people can see
what the answer is :) )
P.S: Cocoa and Xcode are so amazingly simple to use, thank you soooooo
much Apple!!!
--
Al.
☠☢✯☢☠
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden