Re: confusion about memory fragmentation, vm, and blocks
Re: confusion about memory fragmentation, vm, and blocks
- Subject: Re: confusion about memory fragmentation, vm, and blocks
- From: Shawn Erickson <email@hidden>
- Date: Tue, 3 Mar 2009 14:42:43 -0800
On Tue, Mar 3, 2009 at 7:57 AM, Mario Emmenlauer <email@hidden> wrote:
>
> Hi all,
>
> I have a problem understanding memory allocation, blocks and fragmentation.
(at a high level with some hand waving...)
Applications work with/in a virtual memory space. The VM sub-system of
the operating system manages a mapping between the pages in an
applications virtual memory space to pages in some physical storage
device (RAM, page file on disk, etc.).
The amount of memory your application can address at a given time is
limited first by the size of the virtual memory space, minus
reserved/invalid/guard pages, minus pages mapped for shared libraries,
minus pages mapped for use by the OS, etc., minus allocations made by
your code. All of these allocations carve out ranges in your virtual
address space with the unused ranges between them representing
"available" memory to your application.
When you allocate (malloc, etc.) a buffer you get back a pointer to a
contiguous virtual memory range (how the API is defined). If you
attempt to allocate a buffer larger then any of the available
contiguous virtual memory ranges it will fail... despite possibly
having smaller contiguous ranges available that, if contiguous, would
have allow the allocation to succeed.
(note I side stepped "pools" of virtual memory pages that malloc
maintains for small, tiny, etc. allocations to minimize fragmentation
and speed re/allocations)
General it is "hard" to know how much virtual memory your application
has available to it at any given time... likely best to just attempt
to allocate what you when you need it and if that fails back off
and/or purge any data caches your application itself maintains and try
again.
...not knowing exactly your situation...
You could consider using a memory mapped file and slide around a
"window" of contiguous mappings over that file. This would limit/bound
how much virtual memory you use at a given time while giving you
access to an arbitrarily large buffer. ...of course it would be far
easier to switch your application to use a 64b virtual address space.
Anyway try pointing vmmap at an application of your choice to get an
idea about the virtual mappings that exist.
-Shawn
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden