January 2008 - Darwin-kernel

Re: Panic in OSMalloc??
by Michael Smith 11 Jan '08

11 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Jan 10, 2008, at 11:15 PM, Vishal Shetye wrote: Why would you do this? LOCK flag = condition UNLOCK if (flag) thing = allocate() LOCK if (!condition) do_free = true else do_free = false queue_enter(thing) endif UNLOCK if (do_free) free(thing) endif endif = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Could you please explain what are the consequences of calling OSMalloc with some spin lock held. I'm using lck_spin_lock() to lock a global variable before OSMalloc and release it once the item allocated has been added to a queue. Firstly - spinlocks are a low-level primitive; they are not generally suitable for use in high-level code. At the level you are working, you should almost never need a spinlock - use a proper mutex if you must, but strive whenever possible to design your code so that you don't need locking in the first place. Secondly; why are you locking the global and then blocking? You should be using a try/backout scheme to avoid holding the lock while you sleep: This email sent to site_archiver(a)lists.apple.com

1 0

malloc returning NULL, was Panic in OSMalloc??
by Rick Macklem 11 Jan '08

11 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com I just responded to the "Panic in OSMalloc??" thread and then realized that "what to do when malloc returns NULL" is an interesting question, for which I don't have a good answer... When malloc returns NULL, it makes sense that the system is in some sense constipated. To know how much, you'd need to know how the allocators are implemented and what they are being used for. (Beyond what I know about MacOS X.) It seems to me that a non-blocking malloc returning NULL could just indicate that the allocator has hit a transient high water mark and retrying the allocation allowing blocking would make sense? Whether or not retrying a blocking malloc when it returns NULL is a good plan is more questionable. It seems to me that, for this case, the system may be pretty badly constipated and returning ENOMEM may be preferable? My NFS code was written way back when, when the BSD kernel malloc never failed for the blocking case, so I didn't have to deal with this. I'm hoping others will have comments? rick _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

RE: Panic in OSMalloc??
by Rick Macklem 11 Jan '08

11 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Fri, 11 Jan 2008, Vishal Shetye wrote: Hi Brian, Could you please explain what are the consequences of calling OSMalloc with some spin lock held. I'm using lck_spin_lock() to lock a global variable before OSMalloc and release it once the item allocated has been added to a queue. I'm not Brian and won't pretend to be, but... the short version is that you can't block holding a spin lock, as I understand it. (Since any other thread would just be busy waiting in a tight loop waiting for the lock while the thread holding the lock blocked, it wouldn't make much sense, even if the implementation permitted it.) Besides returned NULL value, is there any other difference between OSMalloc() and OSMalloc_noblock()? How I handle these situations (not saying it's the right way, but it is an easy way:-) is: - malloc new structure, blocking as required, until I have one - lock - search list and add new one, as required - unlock - free new one, if it wasn't needed (Yep, this case results in an extraneous malloc/free.) Another easy way (not so easy if multiple lists and structures are involved): tryagain: - lock - search list - if new one needed - if no new one allocated - attempt non-blocking allocation - if new one is allocated - add to list - else - unlock - do blocking allocation of new one - goto tryagain - add to list - unlock - if new one allocated, but not used - free new one Others will probably have more clever approaches. Personally, I'm a big KISS (keep it simple stupid) proponent, unless it is a critical path w.r.t performance. Have fun with it, rick _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… In case OSMalloc_noblock() returns NULL can I try allocation immediately? Probably not a good plan. I'd guess that it will keep returning NULL until some other thread frees up some memory. Since looping on OSMalloc_noblock() would not block, it would hog the CPU. (In a traditional non-preemptive BSD kernel, the system would be hung, but I don't know diddly about the Mach side of things, so I don't know if other Mach threads would still be able to run and eventually release memory.) (Hopefully others with more expertice on the Mach side will chime in.) This email sent to site_archiver(a)lists.apple.com

1 0

RE: Panic in OSMalloc??
by Vishal Shetye 11 Jan '08

11 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:cc:references:subject:date:message-id:mime-version:content-type:content-transfer-encoding:x-mailer:in-reply-to:thread-index:x-mimeole; bh=IEbueiy/9ImYtAUsRxoWESotyDW25axcwNFbaCe/2cM=; b=MJG1ONd7kLAmBIyLhdFJ379FfU13dTR2Q5RBnHM3oOAqHBtGHSRreZZN0fNl79/oHarO8AM18of844Dqe1R9jhcseyIdVlTeuMbQ0mbzJnCKdCKRhp51BKS2Xx+KSHRwigcIfI4LUjEem96Xl7AxGPPYQaOoOpAacrM0lvW6jOU= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:references:subject:date:message-id:mime-version:content-type:content-transfer-encoding:x-mailer:in-reply-to:thread-index:x-mimeole; b=VSMX92rKfK9In5EHvBDzu8/mlqiYTqcD1wxu3ndQH8w+X7DSN+r10e+PW5Ld1Q+OiqpSEqZl1CIUcOkGnKHgvz2xE5CplU95brq9LYxz2Bv7/JsOZPcRvKQ6wC5ep4+wkc+x5ZbPWYrn+5+xpgrOwKvMQQ9RAxU8Pb7km00U+0U= Thread-index: AchGHDamPyVeCb/GRXOxXrxSMAfcOAOAsMbg Hi Brian, Could you please explain what are the consequences of calling OSMalloc with some spin lock held. I'm using lck_spin_lock() to lock a global variable before OSMalloc and release it once the item allocated has been added to a queue. Besides returned NULL value, is there any other difference between OSMalloc() and OSMalloc_noblock()? In case OSMalloc_noblock() returns NULL can I try allocation immediately? ________________________________ Thanks - vishal shetye -----Original Message----- From: Brian Bergstrand [mailto:brian@classicalguitar.net] Sent: Monday, December 24, 2007 4:31 PM To: Vishal Shetye Cc: darwin-kernel(a)lists.apple.com Subject: Re: Panic in OSMalloc?? You should not call OSMalloc while holding any lock. If you must do this, then use OSMalloc_noblock() instead - but be prepared to deal with NULL if the request can't be filled. On Dec 24, 2007, at 8:40 AM, Vishal Shetye wrote: > Hi, > > I've written a kext that listens on fileop scope. I'm using OSMalloc > and OSFree for memory allocation/freeing. > I'm getting "thread_invoke: preemption_level 1" panic. Following is > the backtrace of panic. > OSMalloc() is the last function my kext calls. > An area that I suspect is, > I'm using same "OSMalloc Tag" in different parts of my module which > might call OSMalloc simultaneously. > Do I have to malloc with some locking under such scenarios or is it > handled by OSMalloc() as I can see > calls to mutex_lock() and subsequently to lck_mtx_lock_wait(). > > What might be the case wherein OSMalloc(), trying to wait will > result in such a panic??? > > Backtrace > > #0 Debugger (message=0x3bc46c "panic") at /SourceCache/xnu/ > xnu-792.9.72/osfmk/i386/AT386/model_dep.c:706 > #1 0x00128b5e in panic (str=0x3bcd50 "thread_invoke: > preemption_level %d\n") at /SourceCache/xnu/xnu-792.9.72/osfmk/kern/ > debug.c:202 > #2 0x001350ce in thread_invoke (old_thread=0x29c7c78, > new_thread=0x1b70000, reason=0) at /SourceCache/xnu/xnu-792.9.72/ > osfmk/kern/sched_prim.c:1287 > #3 0x0013572c in thread_block_reason (continuation=0, > parameter=0x0, reason=0) at /SourceCache/xnu/xnu-792.9.72/osfmk/kern/ > sched_prim.c:1692 > #4 0x001357bd in thread_block (continuation=0) at /SourceCache/xnu/ > xnu-792.9.72/osfmk/kern/sched_prim.c:1714 > #5 0x0012f297 in lck_mtx_lock_wait (lck=0x13adcb8, > holder=0x29cae58) at /SourceCache/xnu/xnu-792.9.72/osfmk/kern/ > locks.c:569 > #6 0x00194e4e in mutex_lock () > #7 0x00141242 in zalloc_canblock (zone=0x13adcb0, canblock=1) at / > SourceCache/xnu/xnu-792.9.72/osfmk/kern/zalloc.c:638 > #8 0x0012d780 in kalloc_canblock (size=12, canblock=1) at / > SourceCache/xnu/xnu-792.9.72/osfmk/kern/kalloc.c:256 > #9 0x0012df5f in OSMalloc (size=12, tag=0x2af7600) at /SourceCache/ > xnu/xnu-792.9.72/osfmk/kern/kalloc.c:263 > #10 0x1f5796fb in ?? () > #11 0x1f576c60 in ?? () > #12 0x1f573319 in ?? () > #13 0x0031810a in kauth_authorize_action (scope=0x1c1ca04, > credential=0x26a7204, action=2, arg0=41112608, arg1=4746816, arg2=2, > arg3=0) at /SourceCache/xnu/xnu-792.9.72/bsd/kern/ > kern_authorization.c:405 > #14 0x0031827e in kauth_authorize_fileop (credential=0x26a7204, > action=2, arg0=41112608, arg1=4746816) at /SourceCache/xnu/ > xnu-792.9.72/bsd/kern/kern_authorization.c:550 > #15 0x0032710e in close_internal (p=0x2273000, fd=5, fp=0x1d8e5e0, > flags=3) at /SourceCache/xnu/xnu-792.9.72/bsd/kern/kern_descrip.c:1043 > #16 0x003272ae in close (p=0x2273000, uap=0x25bef0c, > retval=0x25bef50) at /SourceCache/xnu/xnu-792.9.72/bsd/kern/ > kern_descrip.c:985 > #17 0x0036de6c in unix_syscall (regs=0x29c7ef0) at /SourceCache/xnu/ > xnu-792.9.72/bsd/dev/i386/systemcalls.c:201 > #18 0x00197ff3 in lo_unix_scall () > Cannot access memory at address 0xbffffa98 > Cannot access memory at address 0xbffffa9c > Brian Bergstrand <http://www.bergstrand.org/brian/> PGP Key ID: 0xB6C7B6A2 _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to debug VM trouble?
by Michael Smith 09 Jan '08

09 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Jan 7, 2008, at 11:03 PM, Michael Crawford wrote: My problem may be related to the amount of memory in the Mac. Also we've only seen the problem on G5 Macs - Intel Mac Pros work fine. One of our QA guys gets consistent test failures, and one kernel panic on a 5 GB dual G5. The G5 I have has only 512 MB of RAM. I just emailed my manager asking him to authorize the purchase of at least 4 GB more. I understand that the kernel always runs in 32-bit mode, even on machines with large memory that have userspace programs running 64-bit. How exactly is this handled by the kernel? The details are complicated and probably not relevant to your problem. = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… It would help, as I noted in private email, if you were to describe the nature of the data comparison errors you're seeing, as this could help those of us intimately familiar with the sort of trouble you might be having to point you in a useful direction. This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to debug VM trouble?
by Michael Crawford 08 Jan '08

08 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=PfZiVe9Z/nrwsREutJzFE/0a6U8okdI+9cVsw6JPtNA=; b=au8wjkm8a53d+Shu3dvqxYJMhC2KJ08tI3CTMGzLCqtNjar0w3xBeW4S9aAIxNFJaEg6vZv7cIyo7dlkUkwN8dv970E1IuS8CaRdx0LHiF5vceR6olrdmQbtagHQMYRZC/LG7sNNQS2hAoJuFz8cMdfKC7gZe0Xe2eGYviUKkkE= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=af+UWtVyDeRnTxqxdn1MGPw+kX99E38Q7Bive+VrC48pbOhg9UiZwq13UhBSJNHUU+J/H3h5c9PMWUKSKGjz15+jRUdDRq7MQDIc3tTydQC7NijA/2TQndg06Bgl/Wm8PDtUVT4ScByVdBHUMMxp1ALTcbU+C84gqz/uVzio5nU= Hi, Thanks for all your advice. My problem may be related to the amount of memory in the Mac. Also we've only seen the problem on G5 Macs - Intel Mac Pros work fine. One of our QA guys gets consistent test failures, and one kernel panic on a 5 GB dual G5. The G5 I have has only 512 MB of RAM. I just emailed my manager asking him to authorize the purchase of at least 4 GB more. I understand that the kernel always runs in 32-bit mode, even on machines with large memory that have userspace programs running 64-bit. How exactly is this handled by the kernel? Our tiger driver used IOMemoryCursor for PowerPC and IODMACommand for Intel, because the Tiger PPC kernel didn't provide IODMACommand. Our Leopard driver, where we're finding the test failures, uses IODMACommand on both platforms. Thanks for your help! Mike Crawford mdcrawford at gmail dot com http://www.geometricvisions.com/ <-- Free piano music downloads _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to debug VM trouble?
by Andreas Fink 06 Jan '08

06 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On 05.01.2008, at 02:36, Michael Crawford wrote: Hi, We have a stress testing tool for our hardware RAID controller that spawns lots of threads, then writes lots of files and reads them back to verify the data. Under some configurations, which cause heavy VM paging, there are comparison failures. Under other configs that don't stress the VM so much, it works correctly. My coworkers think this could be a bug in Leopard's VM implementation. If that's the case, Apple certainly needs to know about it, but the bug is difficult to reproduce - one has to leave the tool running for hours, for example. Whether it's Apple's bug or mine, I certainly want to find the cause. I'm not sure how to proceed. Can anyone give me some debugging tips? Thanks, Mike -- Michael David Crawford mdcrawford at gmail dot com Enjoy my art, photography, music and writing at http://www.geometricvisions.com/ --- Free Music Downloads --- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/afink%40list.fink.org _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… If its a VM fault of the kernel, it should be reproducible also on standard drives. Can you verify that? And what do you call a comparison failure? The data read is not the same as the data written? My guess is that you do an action on memory which gets paged away but you have not noticed so. So you start reading rubbish. To analyze this, a deeper look into how the code works would bee needed. This email sent to afink(a)list.fink.org This email sent to site_archiver(a)lists.apple.com

1 0

How to debug VM trouble?
by Michael Crawford 05 Jan '08

05 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; bh=14IqyBGzAry3oK/iHcXdSVC7fdvY0jLJtGaFcABION4=; b=b5WS3QPiIHCqazvDvLnr3o8aZtkYhwM1cgmFDgwB4LKYPY1wH6CEVQJMexmReZdW9T6vYJ6rfe+j9Qe6VVzFW1MRUfhtqdrDbJcDyGsDPs6UitE9mgfW4YafvttHm5xy7t4fyJmZwKWi3JsaKWrrAvKpJh1CAR2qTvsonTKgRJs= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=PUDZCSJYs2ODaSSuWQxF55fv6qT01FrwFxGtrmlYv4OaUYHKgltju3cqVn+Wxz8JotD54PIWdYKlEfQaJ31NS4wDbHwTr/T7Tykwmt0BxJ/MByQTXH26JmpEOCKqL5n6v20+sEnCHPGg+NaHwDJsGkIzJXNtztRtCOg9Jd2Bx/I= Hi, We have a stress testing tool for our hardware RAID controller that spawns lots of threads, then writes lots of files and reads them back to verify the data. Under some configurations, which cause heavy VM paging, there are comparison failures. Under other configs that don't stress the VM so much, it works correctly. My coworkers think this could be a bug in Leopard's VM implementation. If that's the case, Apple certainly needs to know about it, but the bug is difficult to reproduce - one has to leave the tool running for hours, for example. Whether it's Apple's bug or mine, I certainly want to find the cause. I'm not sure how to proceed. Can anyone give me some debugging tips? Thanks, Mike -- Michael David Crawford mdcrawford at gmail dot com Enjoy my art, photography, music and writing at http://www.geometricvisions.com/ --- Free Music Downloads --- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Hey I need your help!!
by Hector Martinez 04 Jan '08

04 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.de; h=X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Message-ID; b=nzvAu+U3P8zB/Ys3d8y+rbqhIxAyKdVTVwRLk1kgDhqyGOcyfS8HRMLxA6fL7EXlTHW8sdwHeIYbnoI9UnaYsb4RPJmx6ePPGToDVZkC2RF5DMGE1MkQnABdv+s+Uo6ARZ8YDpnC6YJtPlaENEGloVbVty2kB0dLUZdxvKJSZX0=; _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: missing driver (AppleMacRiscPCI) at startup on G5
by Stephane Sudre 04 Jan '08

04 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On 2 janv. 08, at 10:24, Quinn wrote: _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 19:52 -0800 19/12/07, Al Ciplickas wrote: Thanks again, Terry. The reason I may be 'wasting' time on this issue is that this G5 10.4 system and applications have 3 years of saved mail, other applications and various folders of work. It does not seem easy to transfer all that to a new system partition. The evidence is that I reloaded the 10.4 (by upgrade) and it does not work. There seem to be no tools to move everything BUT system stuff to a new system partition. Migration Assistant (in /Applications/Utilities) can help with this sort of thing. It can help but it will not work correctly for stuff in /Library AFAIK. So 3rd party LaunchDaemons are not moved for instance. This email sent to site_archiver(a)lists.apple.com

1 0