Re: precision problem or my error?
Re: precision problem or my error?
- Subject: Re: precision problem or my error?
- From: Greg Herlihy <email@hidden>
- Date: Fri, 07 Apr 2006 13:33:26 -0700
- Thread-topic: precision problem or my error?
According to limits.h the maximum value that an int can represent is:
#define INT_MAX 2147483647 /* max value for an int */
Since 2,164,362,500 (the product of 50 and 43,287,250) is greater than this
maximum, it cannot be stored in an int (although it could be stored in an
unsigned int because its maximum is 4,294,967,295).
Furthermore, I'm not sure why you think that two bytes contain 32 bits of
data. A byte contains 8 bits of data on both the PowerPC and Intel chips, so
representing a 32-bit value requires four bytes of storage.
Greg
On 4/7/06 12:54 PM, "Ivan Kourtev" <email@hidden> wrote:
> Hi,
>
> I have a problem that is driving me crazy and I cannot identify the
> cause of it. I have a very simple function that computes 100 equally
> spaced portions of an int number (essentially the values of 1%, 2%,
> 3%, ..., 100% of a number). The numbers aren't very big, about
> 20,000,000 to 50,000,000 tops, yet I seem to be running out of
> precision as if int's were 2-bytes long (and not 4). This is
> happening on both a DC G5 Powermac and a G4 PB, with or without 64-
> bit arithmetic checked.
>
> I know, not exactly a Cocoa question, but is is something related to
> Mac's or ObjC runtime that's making this happen? I did check and the
> problem occurs in a plain-C strand-alone program as well.
>
> The function code is at the end. If I call it in a fragment of code
> like this:
> ...
> int x[100];
> foo(x, 100, 43287250 );
> ...
>
> Then the output is like this:
> ...
> 2006-04-07 15:50:08.524 A[4263] j * maxVal = 49 * 43287250 = 2121075250
> 2006-04-07 15:50:08.524 A[4263] a[48] = 21210752
> 2006-04-07 15:50:08.524 A[4263] j * maxVal = 50 * 43287250 = -2130604796
> 2006-04-07 15:50:08.524 A[4263] a[49] = -21306047
> ...
> and all subsequent values of a[] are negative. I checked and 50 *
> 43287250 is indeed where the number crosses the 31-bit which would
> result in a negative number if I were working with 2-byte short int's.
>
> What is going on here? Can anyone shed any light on this? Have I
> found some strange bug in the compiler/libraries?
>
> Thanks for any help,
>
> -- ivan
>
>
>
> void foo( int *a, int parts, int maxVal )
> {
> int i = 0;
> int j = 1;
>
> NSLog( @"sizeof(*a) = %d", sizeof(*a) );
> NSLog( @"sizeof(parts) = %d", sizeof(parts) );
> NSLog( @"sizeof(maxVal) = %d", sizeof(maxVal) );
>
> for ( ; i < parts-1; i++ ) {
> NSLog( @"j * maxVal = %d * %d = %d", j, maxVal, j*maxVal );
> a[i] = ( j * maxVal ) / parts;
> NSLog( @"a[%d] = %d", i, a[i] );
> j++;
> }
> a[parts-1] = maxVal;
> NSLog( @"a[%d] = %d", parts-1, a[parts-1] );
> }
>
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Cocoa-dev mailing list (email@hidden)
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden