Mailing Lists: Apple Mailing Lists
Image of Mac OS face in stamp
Re: ulimit doesn't limit memory usage
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: ulimit doesn't limit memory usage



The setrlimit() administrative limits are not enforced except for certain ones which are enforcible in BSD code. Those which would require enforcement in Mach code (VM-based enforcement or scheduler- based enforcement) are not currently enforced at all.

These administrative limits were never intended to save the system from fragile code; the way to do that is to make the code less fragile. They historically date back to the days when CPU time and memory usage were billable resources, and process accounting and system accounting were used on old timeshare machines that were purchased by groups, such as several University departments, because there was no way for a single University department to be able to afford its own PDP-8. Today, they are basically knobs that have to be there so old code can be ported, but which don't actually have to do anything, for the most part.

The limits are considered administrative, as they are not intended to permit the creation of security policy dependencies, nor are they intended to mitigate problems related to application memory leaks.

The following is the current(0) MacOS X resource limit support table:

    ------------- ---------    ---------
    Limit name    Standard?    Enforced?
    ------------- ---------    ---------
    RLIMIT_CPU      YES          NO
    RLIMIT_FSIZE    YES          YES(1)
    RLIMIT_DATA     YES          NO
    RLIMIT_STACK    YES          YES(1)
    RLIMIT_CORE     YES          YES
    RLIMIT_AS       YES          NO
    RLIMIT_RSS      NO(2)        NO
    RLIMIT_MEMLOCK  NO           NO
    RLIMIT_NPROC    NO           YES(3)
    RLIMIT_NOFILE   YES          YES
    ------------- ---------    ---------

(0) This table is subject to change.

(1) Not entirely correctly enforced; specifically, boundary conditions are clipped before or after the boundary, rather than exactly at it.

(2) RLIMIT_RSS is a MACOS X-specific alias for RLIMIT_AS.

(3) Whether this limit is the upper bounds depends on current sysctl settings on the machine.

See also:

http://www.opengroup.org/onlinepubs/009695399/functions/ setrlimit.html

To determine whether or not a limit is enforced by an OS, rather than using the return code of setrlimit(), you must first use getrlimit() and compare the result to RLIM_INFINITY; if it's RLIM_INFINITY, then you should not set a limit (you can, but you will confuse child processes which call getrlimit() themselves to check for enforcement, if you do).

--

FWIW, the reason your test program bogs down the system is because it starts swapping, so you end up limited to disk speed; the sample program you give is a degenerate case.


If you actually *need* to kill a process when its resource usage gets to a certain point, and you can only make minor modifications to the process, one way to do this would be to start a watchdog thread that sleeps wakes up periodically, and calls kill(getpid(), SIGTERM); if any of the fields from getrusage(RUSAGE_SELF, &rusage); end up geting "too large".


If you need an external watchdog, you'll have to get creative with something like:

char cmd[256];
int rss;
FILE *fp;
int rss_max_allowable = XXX; /* some upper bound */
int pid_to_watch = YYY; /* pid that leaks memory */
...
for(;;) {
sleep(300); /* wake every 5 minutes */
...
sprintf(cmd, "ps -p %d -o rss | tail -1", pid_to_watch);
if ((fp = popen(cmd)) != NULL) {
if (fscanf(fp, "%d", &rss) == 1) {
if (rcss > rss_max_allowable) {
kill(pid_to_watch, SIGHUP); /* "reset" process; may use other signal or SIGKILL */
}
}
pclose(fp);
}
}
...


Obviously, you can use this with anything ps can find out about a process, or any combination of things it can find out. Using the signal system this way lets you be a lot more flexible than the process simply crashing when it hits the limit. For example, maybe you could add a SIGHUP handler that reset the process instead of killing it (this would work with e.g. bind or sendmail, which both use dying children to do memory garbage collection), etc..

Hope that helps...

-- Terry

On Jun 30, 2005, at 2:39 PM, Amy wrote:
Hello.

I'm trying to port an application which uses getrlimit / setrlimit to set the maximum memory the application will use, but they don't seem to actually work. I tried wrapping the application in a script which uses ulimit to limit the maximum memory usage, and that also doesn't seem to do anything :\

Is there any way to limit the maximum memory an application uses, as I'm finding that if you use all physical memory and continue accessing new memory, then Tiger is becomes VERY unresponsive, to the point where I'm having to simply hard-reset in order to regain control. However I would perfer not to rewrite the program as it would be quite hard to keep track of the memory it is allocating and making sure it stays within an acceptable limit.

As a (completely silly) example of what I would like to work, I attach an example program below. I expect this program to quit fairly quickly, but instead it carries on and chews through the entire of virtual memory.

WARNING: Do not compile and run this program if you have anything important running. This is an EXAMPLE of how I'm trying to use setrlimit. I've found that it can cause my computer to (at least appear to) hang, so DON'T run it and then come crying to me!

If anyone could tell me why this program isn't using getrlimit correctly, and if there is a way to correct it, I would be very thankful!

Amy

-- program below --


#include<iostream> #include <unistd.h> #include <sys/time.h> #include <sys/resource.h>

void
set_memory_limits(float size)
{
struct rlimit r;
__typeof__ (r.rlim_cur) limit = (__typeof__ (r.rlim_cur))(size * 1048576);


   // Heap size
   getrlimit(RLIMIT_DATA, &r);
   r.rlim_cur = limit;
   setrlimit(RLIMIT_DATA, &r);
   // Resident set size.
   getrlimit(RLIMIT_RSS, &r);
   r.rlim_cur = limit;
   setrlimit(RLIMIT_RSS, &r);
   // Virtual memory.
   getrlimit(RLIMIT_AS, &r);
   r.rlim_cur = limit;
   setrlimit(RLIMIT_AS, &r);
 }


int main(void) { int i = 0; char* foo; set_memory_limits(10000000); while(foo = (char*)malloc(1000000)) { for(int l=0;l<1000000;l++) foo[l]='a'; std::cout << i++ << std::endl; } }

_______________________________________________
Do not post admin requests to the list. They will be ignored.
Unix-porting mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
40apple.com


This email sent to email@hidden


_______________________________________________ Do not post admin requests to the list. They will be ignored. Unix-porting mailing list (email@hidden) Help/Unsubscribe/Update your Subscription: This email sent to email@hidden
References: 
 >ulimit doesn't limit memory usage (From: Amy <email@hidden>)



Visit the Apple Store online or at retail locations.
1-800-MY-APPLE

Contact Apple | Terms of Use | Privacy Policy

Copyright © 2011 Apple Inc. All rights reserved.