Re: precision problem or my error?
Re: precision problem or my error?
- Subject: Re: precision problem or my error?
- From: Steve Christensen <email@hidden>
- Date: Fri, 7 Apr 2006 13:20:33 -0700
On Apr 7, 2006, at 12:54 PM, Ivan Kourtev wrote:
I have a problem that is driving me crazy and I cannot identify the
cause of it. I have a very simple function that computes 100
equally spaced portions of an int number (essentially the values of
1%, 2%, 3%, ..., 100% of a number). The numbers aren't very big,
about 20,000,000 to 50,000,000 tops, yet I seem to be running out
of precision as if int's were 2-bytes long (and not 4). This is
happening on both a DC G5 Powermac and a G4 PB, with or without 64-
bit arithmetic checked.
I know, not exactly a Cocoa question, but is is something related
to Mac's or ObjC runtime that's making this happen? I did check
and the problem occurs in a plain-C strand-alone program as well.
The function code is at the end. If I call it in a fragment of
code like this:
...
int x[100];
foo(x, 100, 43287250 );
...
Then the output is like this:
...
2006-04-07 15:50:08.524 A[4263] j * maxVal = 49 * 43287250 =
2121075250
2006-04-07 15:50:08.524 A[4263] a[48] = 21210752
2006-04-07 15:50:08.524 A[4263] j * maxVal = 50 * 43287250 =
-2130604796
2006-04-07 15:50:08.524 A[4263] a[49] = -21306047
...
and all subsequent values of a[] are negative. I checked and 50 *
43287250 is indeed where the number crosses the 31-bit which would
result in a negative number if I were working with 2-byte short int's.
No, going negative when bit 31=1 happens with 4-byte ints; 2-byte
ints go negative when bit 15=1.
What is going on here? Can anyone shed any light on this? Have I
found some strange bug in the compiler/libraries?
For the example you gave, 50 * 43287250 is 2164362500 unsigned; in
hexadecimal that's 0x81018D04, which is definitely a negative 4-byte
int. If the range of numbers you're using will go up to about 50M in
100 parts, even unsigned ints won't be big enough (50M * 100 =
5,000,000,000, or 0x12A05F200).
One thing you could do to vastly extend the range would be to use a
64-bit int (long long) for the multiply-divide calculation:
...
long long value = ( j * maxVal ) / parts;
a[i] = value;
...
steve
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden