Mailing Lists: Apple Mailing Lists
Image of Mac OS face in stamp
 
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Strange behaviour of mmap() in OS X

On Sunday, June 22, 2003, at 4:36 AM, Marcel Weiher wrote:
Any pageout algorithm that automatically punishes mapped file pages (simply because they don't have to be cleaned before recycling in many cases) is inherently unfare.

You seem to be making the assumption here that "fairness" is an inherent good. I feel no inherent moral imperative for treating memory pages fairly. I do see an inherent *practical* requirement for a VM system to be stable and predictable without catastrophic failures.

Sorry, I didn't mean to make any moral judgments. I simply stated the page replacement policy. Which is, except where otherwise directed, a global LRU algorithm. For the most part, this works quite well. It also has the benefit of being quite predictable. Where it isn't desired, there are simple ways around it (as I pointed out with msync(), and the no-cache fcntl() options).

Other algorithms, many of which have been (and continue to be considered), tend to make prediction of the policy more difficult for the multimedia software that is the bread and butter of Mac OS X. And lets face it, when you try to run those kinds of applications on a system that is by nature way over-committed in memory resources, being able to predict the default behavior is important.

[mapped pages are already 'punished']

In theory, maybe. In practice, the system will push everything else out of RAM onto disk in favor of the mapped file.

If the current application _needs_ to bring in a bunch of mapped file pages in order to function properly, it better be able to. If that means we had to swap some not-used-recently heaps from other applications - so be it. A little thrash when the user actively switches apps is better than denying an opportunity for an application to run at all in the limited resources available to it. And, again, for many of the key [data intensive] multimedia applications in our key markets, running degraded is worse than not running at all.

to cause faults and detect where the thread "is". But that's assuming a straight [single] linear progression through the pages. With vector code, etc, you often make several passes through small[ish] data chunks and then move on. So, "sequential" isn't all that meaningful these days.

Huh? The assumption you seem to be making here is that with sequential, we need to know exactly when a page is 'done' so we can then immediately discard it. Why? All that is necessary is to know that when memory is tight, you start discarding sequential pages from the start of the sequential region. No clairvoyance required!

Surely you don't mean "and take ALL the sequential pages (including the ones the app hasn't finished with yet) before we take ANY pages that need to be swapped (even though the pages that need to be swapped are MUCH older and not currently needed)?" But anything else does require some clairvoyance again. Again, how far back do we need to keep?

Believe me. Many algorithms that try to guess at the relative importance of "slightly" older sequential pages vs. "much" older swappable pages have been examined. For instance, ones that age sequential pages 2, 3, or even 10 times faster than other [swappable] pages have been tried and STILL tend to push the system over the knee of the swapping curve anyway. Worse, they also tend to cut the knees out from under multimedia applications much sooner (often long before a crisis really exists).

So, after all that, we have found that there are far fewer applications that benefit from those altered algorithms than not. And most of those are much simpler, and can easily be modified to describe their desire to NOT participate in LRU for a particular set of pages. That's certainly much more straightforward that trying to "opt out" of a more complex algorithm for the rest of them (frameworks used in the more complex applications tend not to know which mode their caller would prefer - and can't be altered for legacy reasons).

It sounds like your application fits that first category nicely, as well. And you seem to agree, because you are quite willing to make those advisory calls already. After all this well-mannered debate, it seems it comes down to an issue of you liking the semantics of madvise() for making that indication rather than msync().

I'll see what I can do to make both equally as effective.
--Jim _______________________________________________
darwin-development mailing list | email@hidden
Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-development
Do not post admin requests to the list. They will be ignored.
References: 
 >Re: Strange behaviour of mmap() in OS X (From: Marcel Weiher <email@hidden>)



Visit the Apple Store online or at retail locations.
1-800-MY-APPLE

Contact Apple | Terms of Use | Privacy Policy

Copyright © 2007 Apple Inc. All rights reserved.