Re: 64-bit/shared memory performance
Re: 64-bit/shared memory performance
- Subject: Re: 64-bit/shared memory performance
- From: "Jamil J. Weatherbee" <email@hidden>
- Date: Wed, 9 Apr 2008 00:22:49 -0700
I have grown extremely frustrated with the shm_open()/mmap() interface.
In order to work with a shared memory object greater than 2GB I have
had to "chain" together multiple segments of an object created with
shm_open().
Now, here is my dilemma. I am specifically trying to avoid a
multithreaded client/server architecture as a very much (in theory at
this point) lower performance alternative to this.
I need to:
1) Create a LARGE contiguous shared memory area (i.e. 4GB+) which is
intended to actually be entirely resident in the physical RAM of the
machine.
2) Access the shared memory area for read only by many client hundreds
of processes in a speedy fashion.
Speedy meaning that at present I have to shm_open() and mmap()
together segments of 1GB in order to produce one large 4GB object in
the individual client processes. This works fine
but as soon as I try and READ the object it appears that the
kernel is doing some kind of "COPY ON READ" as you can see in "top"
that it takes about 6 seconds for RSIZE to grow to VSIZE.
I am unclear on why this is happening as I am no VM expert ---
especially since I am using the MAP_FIXED directive in mmap() so that
the addresses are all the same in every processes
virtual address space --- actually this is what makes the
chaining work. I can only imagine that since it is the VM system
writing some kind of kernel mappings.
3) What I need is the fastest, most overhead free manner of directly
accessing a segment of shared memory from multiple processes. These
processes are spawned and die often and quickly. Actually, they are
the result of .php scripts execing a small compiled C program on the
local machine. This would normally be the domain of a SQL server or
the like but this type of data access cannot be efficiently handled
inside of a standard RDBMS architecture. I need top performance so
the data query must be written in C and must have read access to the
data in memory (the 4GB of shared memory is actually only an "index"
to the real data which exists in secondary storage).
Any help/suggestions would be greatly appreciated.
-Jamil
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden