site_archiver(a)lists.apple.com
Delivered-To: darwin-kernel(a)lists.apple.com
On May 31, 2008, at 4:11 AM, Tim Seufert wrote:
= Mike
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.…
Along those lines... do you ever do anything like transcode video,
or use any other software which reads large files? Many free video
encode/transcode programs let the OS cache their input files even
though they're reading the data exactly once. Such programs can put
enormous dynamic pressure on the VM. Whenever some other process
adds a bit of pressure of its own, swap storms are likely.
Such software was a pretty reliable way to bring 10.4 (or earlier)
to its knees. I haven't tried it under 10.5, and my understanding
is that Leopard's buffer cache has been upgraded to better handle
read-once I/O patterns, but it could still be an issue.
The issue here (regarding large file I/O) is a difficult tradeoff in
the VM/buffer cache that Darwin makes differently from some other
systems.
Under Darwin, applications are expected to advise the OS when they
write file data that won't be re-used rapidly; a failure to do this
will cause the system to dedicate cache pages to recently-written file
data in preference to less-recently used application text and data,
which then has to be paged back in. This choice has real performance
benefits in other scenarios, but it does fall down when applications
move large amounts of file data rapidly without telling the system
that it's not useful to cache it.
You're welcome to play armchair architects over the choice of policy,
but bear in mind that it was made (and is regularly being remade)
consciously and with reasons that make good sense to the folks making
the decisions.
This email sent to site_archiver(a)lists.apple.com
site_archiver(a)lists.apple.com
Delivered-To: darwin-kernel(a)lists.apple.com
That would be cool.
<http://developer.apple.com/bugreporter/>
S+E
--
Quinn "The Eskimo!" <http://www.apple.com/developer/>
Apple Developer Relations, Developer Technical Support, Core OS/Hardware
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.…
At 16:43 -0500 1/6/08, Steve Modica wrote:
I imagine that a good chunk of inactive memory is page cache (or
buffer cache. Not sure which apple uses). Can't these pages be
ditched quickly?
Keep in mind that the Mac OS X buffer cache is unified. From the
kernel's perspective, there's no difference between the VM cache and
the file system cache. So your question, as it's written, makes very
little sense. If a page is clean, it can be ditched quickly. If
that page is mapped into one or more address spaces, it will have to
be removed from those address spaces. But the cost of doing that is
small compared to any disk I/O.
Similarly, if a page is dirty, it doesn't matter how the page got
dirty (that is, via file system I/O or via a memory mapping), it will
have to be written to disk (back to a file or, in the case of
anonymous memory, to a VM swap file). The cost of writing it will
completely drown out the cost of unmapping it from memory.
Finally, be aware that on Mac OS X 10.5 we introduced a new, fifth
category of memory, speculative memory, used to hold pages that have
been read from disk speculatively. While the vm_statistics structure
was extended to reflect this (see the speculative_count field), the
various tools haven't been updated to display it (including Activity
Monitor <rdar://problem/5522515> and vm_stat
<rdar://problem/5522568>). These tools currently lump speculative
pages in with the free pages.
(while I'm at it, I sure wish there was a tool around that would
show the size and contents indexed by inode of the buffer cache)
This email sent to site_archiver(a)lists.apple.com
site_archiver(a)lists.apple.com
Delivered-To: darwin-kernel(a)lists.apple.com
<Dropping lots of the other stuff here>
Steve
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.…
I imagine that a good chunk of inactive memory is page cache (or
buffer cache. Not sure which apple uses). Can't these pages be ditched
quickly? (while I'm at it, I sure wish there was a tool around that
would show the size and contents indexed by inode of the buffer cache)
This email sent to site_archiver(a)lists.apple.com