Re: Is there a 1GB memory ceiling for vm_allocate?
Re: Is there a 1GB memory ceiling for vm_allocate?
- Subject: Re: Is there a 1GB memory ceiling for vm_allocate?
- From: " Kuon - Nicolas Goy - 時期精霊 (Goyman.com SA) " <email@hidden>
- Date: Wed, 24 Jan 2007 08:54:46 +0100
On Jan 23, 2007, at 11:52 PM, John Bullitt wrote:
On Jan 23, 2007, at 5:32 PM, Shawn Erickson wrote:
The VM size isn't the issue really the issue... 32b tasks have a 4
GiB
address space.
...
Thanks Shawn. That's a very helpful and clear explanation.
I'm still a bit puzzled, though. Do you know how other 32-bit apps
handle long vectors? For example, sound or video editing tools that
can be used to edit long (hour-long, multi-tracked, multi-GB-sized)
chunks of data? Are they just paging small chunks of data to and
from the hard disk?
John
We have an app (not on OSX but that's for the concept) which uses
huge amount of memory. It runs on SPARC 64 bit and we uses about
60gig of memory.
Even if you requirement are less, the concept is the same (with your
own approach of course, and in 32bit).
We uses plain C and we do something like this.
#define BLOCK_SIZE 1024 * 1024 * 50 /* block size is 50mb */
#define MAX_BLOCKS 1024 /* We allocate up to 50 gig of memory */
typedef struct {
char *data;
} Block;
typedef struct {
Block **blocks;
int blockCount;
int dataSize;
} Blocks;
This is the general "idea" on own to store huge data. Now, I have
little experience in C++, but in theory, you must write your own
vector class, to allow the use of the above concept.
In short, we have:
Blocks *myMalloc(int size) {
// Check MAX_BLOCKS
...
// Allocate a block with something like
Blocks * b = malloc(sizeof(Blocks));
b->blockCount = size;
b->blocks = malloc(size * sizeof(Block*));
int i;
for(i = 0; i < size; i++) {
b->blocks[i] = malloc(sizeof(Block));
// You decide when you allocate memory
//b->blocks[i]->data = malloc(BLOCK_SIZE * sizeof(char));
}
return b;
}
and
void storeData(Blocks *b, char *data, int dataSize) {
// On which block should we store that data?
int i, l;
for(i = 0, l = b->dataSize; i < MAX_BLOCKS; i++) {
if(l > BLOCK_SIZE)
l -= BLOCK_SIZE;
else {
Block * bb = b->blocks[i];
// we can store dataSize - BLOCK_SIZE in bb->data[l]
// we must store the rest in an other block
}
}
}
Ok, this is for the concept. I wrote this now so on the fly (can't
copy past, closed source still exists).
If you need help, I wrote a high performances blocks management
system in C, and maybe I can help.
Best regards
--
Kuon
Programmer and sysadmin.
"Computers should not stop working when the users' brain does."
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Xcode-users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden