December 2008 - Darwin-dev - lists.apple.com

Re: what is expected malloc behavior re. limits/insufficient resources?
by IainS 24 Dec '08

24 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com LOL .. getting my darwins and OSXs mixed. this is understood - the pointer is harmless until used.... unusable - generally to the extent that a hard reboot is required. (a) Is malloc() supposed to honor ulimits? Is there any official answer to these? Thanks, Iain _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On 24 Dec 2008, at 22:47, Terry Lambert wrote: On 10.{8,9} You probaly meant an earlier version; if not, what year is it and are there flying cars yet? 8) it seems that malloc() returns a non-NULL pointer (i.e. without error). It returns a pointer into a page mapping in the process virtual address space. No physical pages are allocated until you touch them. This can exceed total RAM plus swap because of memory overcommit, which has pretty much been SOP on unix since the DEC VAX version introduced virtual memory. I'm sure you're right --- but these platforms must have had some way of saving you --- I don't remember having this trouble ever on a VAX or a Sun (or even a Prime)... ;-) Subsequent use of the returned pointer can (and does) bring the system down Define "down". once you see a "no free paging space left" message thing are looking grim. Yes the system will spend most of its time attempting to clean pages for you to satisfy your applications demands. AFAICT any process that tries to request memory after the system has run out of paging space is put to sleep. Effectively, unless one can kill the process that's oversubscribed the memory, the system is deadlocked (since, these days pretty much every program relies on malloc) Even something simple like the display sleeping is enough to require a hard-reboot - since one cannot ssh (no new process can be started)... etc. etc. (b) Is malloc() supposed to allow requests in excess of available system resources? This behavior is troublesome - because a simple non-privileged program compiled -m64 can easily stop the system non-privileged programs ought not to be able to make the system unusable (IMO). (with loss of any data that depends on a program that depend on malloc --- i.e. pretty much all of them) does any of the behavior described constitute a bug -- or perhaps an enhancement ? If it cannot be regarded as a bug, perhaps honoring ulimits or providing RLIMIT_VMEM would be a very useful enhancement? This email sent to site_archiver(a)lists.apple.com

1 0

Re: what is expected malloc behavior re. limits/insufficient resources?
by Terry Lambert 24 Dec '08

24 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hi, In a situation where the malloc request: is < MALLOC_ABSOLUTE_MAX_SIZE but > [total RAM + total free swap partition space] or > ulimit -v , or ulimit -d (regardless of soft/hard) Define "down". ---- (a) Is malloc() supposed to honor ulimits? ---- cheers and season's greetings, Iain _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/tlambert%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Dec 24, 2008, at 6:37 AM, IainS <developer(a)sandoe-acoustics.co.uk> wrote: On 10.{8,9} You probaly meant an earlier version; if not, what year is it and are there flying cars yet? 8) it seems that malloc() returns a non-NULL pointer (i.e. without error). It returns a pointer into a page mapping in the process virtual address space. No physical pages are allocated until you touch them. This can exceed total RAM plus swap because of memory overcommit, which has pretty much been SOP on unix since the DEC VAX version introduced virtual memory. Subsequent use of the returned pointer can (and does) bring the system down once you see a "no free paging space left" message thing are looking grim. Yes the system will spend most of its time attempting to clean pages for you to satisfy your applications demands. Interposing malloc debug puts up the appropriate "insane request" message - but under 'normal' malloc the program runs on happily until the system dies. (b) Is malloc() supposed to allow requests in excess of available system resources? ( I would be amongst the first to agree that providing a robust solution is non-trivial -- but... that doesn't mean it isn't worth trying). - There doesn't appear to be a RLIMIT_VMEM so that control is not avail. This behavior is troublesome - because a simple non-privileged program compiled -m64 can easily stop the system (with loss of any data that depends on a program that depend on malloc --- i.e. pretty much all of them) does any of the behavior described constitute a bug -- or perhaps an enhancement ? This email sent to tlambert(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: ipfw source and syslog
by Justin C. Walker 24 Dec '08

24 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Dec 24, 2008, at 08:02 , Jeremy wrote: Look in the network_cmds "project". Justin -- Justin C. Walker, Curmudgeon-At-Large, Director Institute for the Enhancement of the Director's Income -------- The path of least resistance: it's not just for electricity any more. -------- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Trying to find the source for the implementation of ipfw included in OS X 10.5. This email sent to site_archiver(a)lists.apple.com

1 0

Re: ipfw source and syslog
by Iceberg-Dev 24 Dec '08

24 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:in-reply-to:references :mime-version:content-type:message-id:cc:content-transfer-encoding :from:subject:date:to:x-mailer; bh=8nrQ8BDJtb+Rp2++fIc6pGz5sWNgnjmpi2r0tD4N2xc=; b=DXwMxbkyT1s48Y8vulpKKngo+iBoe9p0szW7eT0wvE2+lQ529V1aYXwyGck6bZrQ84 YNBjnYxxtQTPuFyTcpVj3dCNbdnQ9vQhAG0gaHHQBfj87vzegEr2tVwpZy+kzzusup7g 0ozx/1xWVYVP3FY7b1s4NpS/muO0z9S3hYsEE= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=in-reply-to:references:mime-version:content-type:message-id:cc :content-transfer-encoding:from:subject:date:to:x-mailer; b=kUufL8zostE1I9TJbdfJHOx64aeOzxppc9QyH1/8Day7xvHmE9ajXrT/5JiaoZ5ORq Z54BsVe/DIG6NMkI3BqYEnr8xrUeoj4vNt00IO37s/zQre5/8eJfiEjy/nThWOE0O2Wg of4fgkZzxcrOprU2TGMUPdgVeIleXeDur238w= On Dec 24, 2008, at 5:02 PM, Jeremy wrote: vs. ipfw has been integrated into the xnu kernel as far as I know. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Trying to find the source for the implementation of ipfw included in OS X 10.5. Specifically I'm trying to find out what syslog facility ipfw is supposed to log to. The manpage for ipfirewall(4) states that it logs to LOG_SECURITY, as does the code in ip_fw2.c from the FreeBSD site (http:// www.freebsd.org/cgi/cvsweb.cgi/src/sys/netinet/ip_fw2.c) According to /usr/include/sys/syslog.h the LOG_SECURITY facility is aliased to LOG_AUTH, so messages should be available there. However, none of the above information seems to match reality. When net.inet.ip.fw.verbose is set to 1 then ipfw messages are sent to the facility LOG_KERN at the debug level. When net.inet.ip.fw.verbose is set to 2 then ipfw messages are sent to / var/appfirewall.log in a slightly non-standard format. Example: "Dec 15 13:55:17 hostname Firewall73: 65534 Deny TCP x.x.x.x:51182 x.x.x.x:9 in via en0" "Dec 15 12:25:18 hostname kernel[0]: ipfw: 65534 Deny TCP x.x.x.x: 45768 x.x.x.x:1002 in via en0" So it appears that the source on the FreeBSD site is not that which is in use in OS X. Does anyone know where the actual in use ipfw source can be acquired? This email sent to site_archiver(a)lists.apple.com

1 0

ipfw source and syslog
by Jeremy 24 Dec '08

24 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:mime-version:content-type; bh=gCUD8j9745XwILAxIapoPfjsWvCpHj8vUvsXT73lt/M=; b=CjyKgeo6tDAtn9SXe1bhLfoKbhWrm6s03ISM9M1+0aT2c6LkOS6z/klfzc+MHggJ7C DN+PjzYmsfA5797qPfet7dexDnKORdig1VgtPAWPEhlLe+pL/FuhpN/yUL+ypGnATFsi DRloyT/itRVTXU4At9Ng8wZGvVSMjb9c7DJWA= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type; b=vF/KDIctuNys8hwe3ojZbglpXsK5TthbVAFxWXnLxYjcfL8IAzvws4Dy/6MQbGLKhn m3Kc/mfAWXBnxXHdOPyLnu5DSvgHXa19/tZF+aj19p8kaK9P2OdUGP+07P69KZW0HWTl 4P1gL4NwzKVgxZIy9Y55Xsau4VnJjKWZ64Tt0= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

what is expected malloc behavior re. limits/insufficient resources?
by IainS 24 Dec '08

24 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hi, On 10.{8,9} In a situation where the malloc request: is < MALLOC_ABSOLUTE_MAX_SIZE but > [total RAM + total free swap partition space] or > ulimit -v , or ulimit -d (regardless of soft/hard) it seems that malloc() returns a non-NULL pointer (i.e. without error). --- ---- (a) Is malloc() supposed to honor ulimits? - There doesn't appear to be a RLIMIT_VMEM so that control is not avail. ---- cheers and season's greetings, Iain _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Subsequent use of the returned pointer can (and does) bring the system down once you see a "no free paging space left" message thing are looking grim. Interposing malloc debug puts up the appropriate "insane request" message - but under 'normal' malloc the program runs on happily until the system dies. (b) Is malloc() supposed to allow requests in excess of available system resources? ( I would be amongst the first to agree that providing a robust solution is non-trivial -- but... that doesn't mean it isn't worth trying). This behavior is troublesome - because a simple non-privileged program compiled -m64 can easily stop the system (with loss of any data that depends on a program that depend on malloc --- i.e. pretty much all of them) does any of the behavior described constitute a bug -- or perhaps an enhancement ? This email sent to site_archiver(a)lists.apple.com

1 0

Re: The hurdle to get started with Apple's Open Source distributions
by Martin Costabel 23 Dec '08

23 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Thunderbird 2.0.0.18 (Macintosh/20081105) Hello people, Besides the macosforge project, there are mailing lists. <http://lists.macosforge.org/pipermail/darwinbuild-dev/> Instructions for getting started, are for example here: <http://lists.apple.com/archives/darwin-dev/2007/Nov/msg00063.html> <http://www.puredarwin.org/developers/darwinbuild> -- Martin _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Gerben Wierda wrote: I started out looking at Apple's Open Source because I was thinking about patching a bug in the (rather old) jabber server that comes with OS X Server. So, as an ADC member, I downloaded the ChatServer download. But it turns out it does not compile just like that. I started looking for some newbie information, say a cookbook to get things started. I found one about rolling your own XCode project around some PD software, but that was not it. I looked at mailing list archives, but was soon lost in lists, some of which have their last post in 2004 or so. You might want to look at DarwinBuild <http://darwinbuild.macosforge.org/>. This is more or less indispensable for building the open source parts of OSX. I don't know if there are any "good instructions" out there, but it is possible to find out how to use it. This email sent to site_archiver(a)lists.apple.com

1 0

The hurdle to get started with Apple's Open Source distributions
by Gerben Wierda 23 Dec '08

23 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hello people, Yours, G I started out looking at Apple's Open Source because I was thinking about patching a bug in the (rather old) jabber server that comes with OS X Server. So, as an ADC member, I downloaded the ChatServer download. But it turns out it does not compile just like that. I started looking for some newbie information, say a cookbook to get things started. I found one about rolling your own XCode project around some PD software, but that was not it. I looked at mailing list archives, but was soon lost in lists, some of which have their last post in 2004 or so. I decided to get on this list to ask the question: where do I find good instructions that will help me in getting the (jabberd) ChatServer from Apple's Public source compiled and installed on my OS X Server (10.5.6)? Can somebody point me to the information I need? On my own, I have not been able to find it and I have gotten lost in all the relevant and sometimes outdated information. PS. I am looking for updating my stuff, not solutions like macports, fink or such where you end up with a second or even third hierarchy of programs. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to use timeout in semaphore
by Eli Bach 22 Dec '08

22 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-mailer; bh=iOWbeJT83IYhAimNhtRU/29CdQzlaMAIhdE9gbz3W7E=; b=PxrfdUYvN+bxeo3PhzE/aUSF0ShVNQxYkU+SuEK7cy4ojhDbbxd7vshY5qVfbfw5DB YZpGJtqRnA1oJ4/TWPf8ptW/0CDLhrhnHz9+d9N+2KlG1CD5enCJr533c3HkWTMU/MSv ljFL4szxixu7aXuDcpFgwQGmSR8i0bL3oWaXE= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-mailer; b=XmPHE5AFXYCCG6rYwfrSkABGQ7G+8Vo41jmuoTz9jjI5LuubLyfUBWPowYZN5dxjz7 Drs4kleRN4qTmnqYl9ll6acbXPsxtZwyh2hhp+ieFkFW+X3htLneKbeL2JfmEuPhwhB9 JEURcESoROfvfoNR+H8y9vtZXbmkZmyohRG/w= On Dec 21, 2008, at 11:00 PM, Rakesh Singhal wrote: Hi all I am using semaphore using semget, semop, semctl APIs. I have written 2 APIs similar like semwait and sempost using semop. Now I want to use semwait but it is blocking call and I want to get timeout in some defined time if there sempost is not called. Please suggest me. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… >From looking at the man page for sem_wait(), I would suggest setting a timer to send a signal to the process, and then noticing that sem_wait exitted because of the signal and not because the semaphore was released. It is left as an exercise for the reader to determine what signal to best to send, handling case where the signal interrupts other system calls, etc... This email sent to site_archiver(a)lists.apple.com

1 0

How to use timeout in semaphore
by Rakesh Singhal 22 Dec '08

22 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:mime-version:content-type:content-transfer-encoding :content-disposition; bh=wZNexN3Nin+d8OiD7Pm9ogu5+5CZtc7LTPRHEAynFf8=; b=C8A/w4nJPssx52sdaQL91PhnXL/Ww4FFvYcC50xZJoy7Ljr0GhkHsJcmJ/MM0lOLK5 2QPJPk8hrfbTG/bYNpBdxqr02U1uFHlTgzYiU6n0TvGNf8EvSlxHhrmxeQXRwH511iw/ upOGyQQjQnoAWtEdukk9J9BVkQOB0JMRi0PS4= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type :content-transfer-encoding:content-disposition; b=nCpAgqk5QyzzrvItpfpHfTCAxDzY1S6gMIuEGGD30nvhTdVTfpBPemGCXKpHDjpzoR s3Nv25Iu9GK/Hz3ieEuSElRTp+eth58HzNNmEFlXhdnFJDzS5bbUnIVGo2DoQKJ9tyVg l0dm0PUWBDyktubQVCwWMifiDTunivRNPs+Rw= Hi all I am using semaphore using semget, semop, semctl APIs. I have written 2 APIs similar like semwait and sempost using semop. Now I want to use semwait but it is blocking call and I want to get timeout in some defined time if there sempost is not called. Please suggest me. Regards rksinghal _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0