64 bit development -- Is mmap() of over 4GB possible on OS X 10.4/10.5?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

64 bit development -- Is mmap() of over 4GB possible on OS X 10.4/10.5?

Subject: 64 bit development -- Is mmap() of over 4GB possible on OS X 10.4/10.5?
From: "Jamil J. Weatherbee" <email@hidden>
Date: Mon, 10 Mar 2008 00:59:06 -0700

Gentlemen,

I have had the following experience:

No where is it made clear why the size of data you can mmap() is limited in a 64-bit environment (-arch x86_64 or -arch ppc64). For instance I cannot mmap() more than about 3.5GB on my Intel Xeon XServe with 8GB of physical RAM (running 10.5), the same app cannot mmap() more than about 2GB on my G5 XServe with 8GB of physical RAM (running 10.4). I can however locally allocate and use an array like char x[6*1024*1024*1024]; on both and compile and execute without a problem.

I want to use shared memory with a very large data object that will have multiple unrelated readers (process wise) and a single writer. I will be synchronizing access using flock() when I need to write but most of the time the data will be read only. Here is some sample code I wrote to test the mmap() subsystem --- I am using shm_open() because I do not want backing store for this data it will be rebuilt from a separate data store upon reboot.

Am I somehow misusing mmap() by wanting to have one large logically contiguous shared memory space to store this data? I can imagine this has come up in HPC and database engines before, so I am surprised that I cannot find any documents that address it.

Here is some very quickly hacked together sample code and the output it generates for me on a powerPC G5 XServe with 8GB of RAM.

(I know the actual physical memory doesn't matter except in the amount of time you are going to be waiting for pageouts while memset runs)

Try increasing MEMSIZE until it breaks for you and you get 0xffffffffffffffff returned from mmap. BTW - this leaves 10GB of garbage around in the VM if you run it successfully.

What I ideally want is to map one shared 6GB area, for my purposes if I need more virtual memory then I will just increase the physical memory size as the

XServes are now accepting up to 32GB of RAM.

compiled with "gcc -arch ppc64 -o openmem openmem.c"

#include <stdio.h>

#include <sys/mman.h>

#include <fcntl.h>

#include <unistd.h>

#include <string.h>

#include <stdlib.h>

#include <sys/errno.h>

#include <sys/types.h>

#define MEMSIZE ((long)2*1024*1024*1024)

int main (void)

{

int fd1, fd2, fd3, fd4, fd5, err;

char *mem1, *mem2, *mem3, *mem4, *mem5;

printf ("sizeof(void *) = %d\n", sizeof(void *));

printf ("sizeof(int) = %d\n", sizeof(int));

printf ("sizeof(long) = %d\n", sizeof(long));

printf ("memsize = %ld\n", MEMSIZE);

err = shm_unlink ("bank1");

printf ("shm_unlink bank1 = %d\n", err);

err = shm_unlink ("bank2");

printf ("shm_unlink bank2 = %d\n", err);

err = shm_unlink ("bank3");

printf ("shm_unlink bank3 = %d\n", err);

err = shm_unlink ("bank4");

printf ("shm_unlink bank4 = %d\n", err);

err = shm_unlink ("bank5");

printf ("shm_unlink bank5 = %d\n", err);

fd1 = shm_open ("bank1", O_RDWR|O_CREAT);

printf ("fd1 = %d\n", fd1);

fd2 = shm_open ("bank2", O_RDWR|O_CREAT);

printf ("fd2 = %d\n", fd2);

fd3 = shm_open ("bank3", O_RDWR|O_CREAT);

printf ("fd3 = %d\n", fd3);

fd4 = shm_open ("bank4", O_RDWR|O_CREAT);

printf ("fd4 = %d\n", fd4);

fd5 = shm_open ("bank5", O_RDWR|O_CREAT);

printf ("fd5 = %d\n", fd5);

err = ftruncate (fd1, MEMSIZE);

printf ("ftruncate bank1 = %d\n", err);

err = ftruncate (fd2, MEMSIZE);

printf ("ftruncate bank2 = %d\n", err);

err = ftruncate (fd3, MEMSIZE);

printf ("ftruncate bank3 = %d\n", err);

err = ftruncate (fd4, MEMSIZE);

printf ("ftruncate bank4 = %d\n", err);

err = ftruncate (fd5, MEMSIZE);

printf ("ftruncate bank5 = %d\n", err);

mem1 = mmap (NULL, MEMSIZE, PROT_READ|PROT_WRITE, MAP_SHARED, fd1, 0);

printf ("mmap bank1 = %p\n", mem1);

mem2 = mmap (NULL, MEMSIZE, PROT_READ|PROT_WRITE, MAP_SHARED, fd2, 0);

printf ("mmap bank2 = %p\n", mem2);

mem3 = mmap (NULL, MEMSIZE, PROT_READ|PROT_WRITE, MAP_SHARED, fd3, 0);

printf ("mmap bank3 = %p\n", mem3);

mem4 = mmap (NULL, MEMSIZE, PROT_READ|PROT_WRITE, MAP_SHARED, fd4, 0);

printf ("mmap bank4 = %p\n", mem4);

mem5 = mmap (NULL, MEMSIZE, PROT_READ|PROT_WRITE, MAP_SHARED, fd5, 0);

printf ("mmap bank5 = %p\n", mem5);

memset (mem1, 0xAA, MEMSIZE);

memset (mem2, 0xBB, MEMSIZE);

memset (mem3, 0xCC, MEMSIZE);

memset (mem4, 0xDD, MEMSIZE);

memset (mem5, 0xEE, MEMSIZE);

}

Output:

sizeof(void *) = 8 sizeof(int) = 4 sizeof(long) = 8 memsize = 2147483648 shm_unlink bank1 = 0 shm_unlink bank2 = 0 shm_unlink bank3 = 0 shm_unlink bank4 = 0 shm_unlink bank5 = 0 fd1 = 3 fd2 = 4 fd3 = 5 fd4 = 6 fd5 = 7 ftruncate bank1 = 0 ftruncate bank2 = 0 ftruncate bank3 = 0 ftruncate bank4 = 0 ftruncate bank5 = 0 mmap bank1 = 0x6008000 mmap bank2 = 0x8ff4e000 mmap bank3 = 0x10ff4e000 mmap bank4 = 0x18ff4e000 mmap bank5 = 0x20ff4e000

 _______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:

This email sent to email@hidden

Follow-Ups:
- Re: 64 bit development -- Is mmap() of over 4GB possible on OS X 10.4/10.5?
  - From: William Kucharski <email@hidden>

Prev by Date: Re: dyld 2 questions
Next by Date: Re: 64 bit development -- Is mmap() of over 4GB possible on OS X 10.4/10.5?
Previous by thread: Re: dyld 2 questions
Next by thread: Re: 64 bit development -- Is mmap() of over 4GB possible on OS X 10.4/10.5?
Index(es):
- Date
- Thread