[SOLVED] Re: precision problem or my error?
[SOLVED] Re: precision problem or my error?
- Subject: [SOLVED] Re: precision problem or my error?
- From: Ivan Kourtev <email@hidden>
- Date: Fri, 7 Apr 2006 16:20:36 -0400
It was my error (as already noted), and my apologies for posting here
before looking into it more carefully. I don't know what I was
thinking when I was counting these bytes... :(
--
ivan
On Apr 7, 2006, at 3:54 PM, Ivan Kourtev wrote:
Hi,
I have a problem that is driving me crazy and I cannot identify the
cause of it. I have a very simple function that computes 100
equally spaced portions of an int number (essentially the values of
1%, 2%, 3%, ..., 100% of a number). The numbers aren't very big,
about 20,000,000 to 50,000,000 tops, yet I seem to be running out
of precision as if int's were 2-bytes long (and not 4). This is
happening on both a DC G5 Powermac and a G4 PB, with or without 64-
bit arithmetic checked.
I know, not exactly a Cocoa question, but is is something related
to Mac's or ObjC runtime that's making this happen? I did check
and the problem occurs in a plain-C strand-alone program as well.
The function code is at the end. If I call it in a fragment of
code like this:
...
int x[100];
foo(x, 100, 43287250 );
...
Then the output is like this:
...
2006-04-07 15:50:08.524 A[4263] j * maxVal = 49 * 43287250 =
2121075250
2006-04-07 15:50:08.524 A[4263] a[48] = 21210752
2006-04-07 15:50:08.524 A[4263] j * maxVal = 50 * 43287250 =
-2130604796
2006-04-07 15:50:08.524 A[4263] a[49] = -21306047
...
and all subsequent values of a[] are negative. I checked and 50 *
43287250 is indeed where the number crosses the 31-bit which would
result in a negative number if I were working with 2-byte short int's.
What is going on here? Can anyone shed any light on this? Have I
found some strange bug in the compiler/libraries?
Thanks for any help,
-- ivan
void foo( int *a, int parts, int maxVal )
{
int i = 0;
int j = 1;
NSLog( @"sizeof(*a) = %d", sizeof(*a) );
NSLog( @"sizeof(parts) = %d", sizeof(parts) );
NSLog( @"sizeof(maxVal) = %d", sizeof(maxVal) );
for ( ; i < parts-1; i++ ) {
NSLog( @"j * maxVal = %d * %d = %d", j, maxVal, j*maxVal );
a[i] = ( j * maxVal ) / parts;
NSLog( @"a[%d] = %d", i, a[i] );
j++;
}
a[parts-1] = maxVal;
NSLog( @"a[%d] = %d", parts-1, a[parts-1] );
}
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden