August 2009 - Darwin-kernel

Re: Authenticated communication between KEXT and a userland client
by Michael Smith 19 Aug '09

19 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Authenticated communication between KEXT and a userland client
by Andreas Guðmundsson 19 Aug '09

19 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type:content-transfer-encoding; bh=ypJqw8w7C0yvPCWpNeWTAfYc3z6p2GLUfwNyWOFOEo4=; b=PtzfNuqgSLxmJpPSYEBBKmH2N+zXBPmpwuuX17knlwSX11Fo4ZApL9ujW6dAVvjGzP Din+YPdDFI+ROGt8I/5AxkyZXcieV+Rw/3xfdjWEK3IwmPWPYyIykETWHZ8W3EVFaWVU bgI9HleI4M0iElML9mEHoPmkkFsNWQldf+Y8Q= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; b=dOUk/UjUqnI2r07ZUEJW25Lg9eF+jgHeJZHjbbPL6+kQq0b0P1pyOtgO6+WYH9/pUj P+buwubtr4BqnsrnXf5yAdDGtHwPiudMNqlvzf7Mv4e3Zdu7+b312VXQRWcFrsvFOnRD Kg1e3cMchelSWKVGsv6vV8ShijRoGmEcd8GYI= Hi, I have a kext and a userland client communicating with sockets. Now I'd like to be sure that I'm communicating with the correct client so I want the kext to authenticate the client. How can this be done properly? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Memory Management Debugging Tools?
by Duane Murphy 17 Aug '09

17 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 I'm having a problem with an IOKit kext that I hope I would be able figure out using some typical memory management tools. Are there any special memory management tools or options (ie MallocDebug) that can help in the kernel. The problem only occurs at early startup (before Ethernet so I would have to debug with FireWire). It is random but consistent. That is, the panic doesn't always occur, but when it does it is always exactly the same. The panic always occurs in IOUSBControllerV3::HandlePowerChange(). My kext is not anywhere in the stack trace. Decoding the panic shows that the workloop in the CommandGate for the IOUSBControllerV3 instance is NULL. This causes IOEventSource::openGate() to panic. I'm looking for ideas for how to track down this problem. My suspicion is that I have some code that is clearing data in a stale pointer. However, I am open to other possible reasons for causing this problem. Thank you for any tips or ideas, ...Duane - --- 0x12b4c6 <panic+422>: mov 0x4dd280,%eax 0x1ab0fe <kernel_trap+1450>: add $0xcc,%esp 0x1a1713 <trap_from_kernel+26>: mov %edi,%esp 0x4263e7 <IOEventSource::openGate()+9>: mov (%eax),%edx 0x42752f <IOCommandGate::runAction(int (*)(OSObject*, void*, void*, void*, void*), void*, void*, void*, void*)+271>: add $0x3c,%esp 0x86221fe7 <IOUSBControllerV3::HandlePowerChange(unsigned long)+67>: leave 0x86221604 <IOUSBControllerV3::setPowerState(unsigned long, IOService*) +350>: xor %eax,%eax 0x41c606 <IOService::driverSetPowerState()+132>: mov %eax,%edi 0x41c99d <IOService::pmDriverCallout(IOService*)+37>: jmp 0x41c9b5 <IOService::pmDriverCallout(IOService*)+61> 0x13ebde <thread_call_thread+233>: movl $0x0,0x4(%esp) 0x1a14fc <call_continuation+28>: add $0x10,%esp -----BEGIN PGP SIGNATURE----- Version: PGP Desktop 9.10.0 (Build 500) Charset: US-ASCII wsBVAwUBSomLfErg9acQ4r2CAQjqcgf/XMCQvsc78WV5r2oRQN3AQPqp0dVJwJ7q CMMmoCCsEG37t/VsENv2iHT3pv2MXdeHQMGnQ0/ApLtvIfpfWT5PlVWp/F28ZaY8 hyYJKFNkGPazN6mGn+2xGEWq5T5zez0+GiMcW2/MuaW/w8SNCphV14kjj/jr7LiU 7qdzcshSxaqSShryzouIRp/FakOICPuwZ89fS5XFR0lcyUkH2nkDA43d3ajBvHaJ MK6nhjkoz3wFprsrjT/L+Zjib8V/A12XmIKvkX0Hz/cqQdxrdTJBbyIXqdmnQa2B FPU3xQbdibsKDNTwI9+JTsZWmNsqaUJ28+EVym6dzdKCkF3GtAsyyw== =0QyA -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: getPhysicalSegment out of 32b range kernel panic
by Michael Smith 15 Aug '09

15 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

getPhysicalSegment out of 32b range kernel panic
by Duane Murphy 14 Aug '09

14 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 I'm getting a kernel panic indicating getPhysicalSegment out of 32b range 0x101bee000 len 0x1000 in a disk filter kext. The filter allocates a block of memory equal to the size of the block being read or written on each read or write access. I am allocating the memory using IOBufferMemoryDescriptor::withOptions( directionFromTheSourceBuffer | kIOMemoryUnshared, lengthFromTheSourceBuffer ); At some later time when the memory is handed to the lower level driver, getPhysicalSegment() is called on this buffer and the address is out of range. Is there a way to control the memory allocation so that it is always below the 32b range? Other suggestions? Thanks, ...Duane -----BEGIN PGP SIGNATURE----- Version: PGP Desktop 9.10.0 (Build 500) Charset: US-ASCII wsBVAwUBSoWVkUrg9acQ4r2CAQiA7Qf7Bi2kK1QZ8fuhL6IYYIdkFbtb5bWLc3u9 gUXDUj4ySGYMdmUpXKPFRTlIrmv9zy6zIiWVj9PHd4EtWQAbqBL7ujLbotEMtYZb qwr/Lo9/3BZogxP1JkttVDJz7WG+4foD+bV8AAbnxG3NCadIPMIIAtVebNhuu4cT LeoC3k1rq3qxO4oSPFzpIVq2BQnAriKCBLAhhH8JK2msN1FvyhM8Z+oTYupUsl9o 1vzGxvoqjJhAOg6Y9ULNf37y4qdPk4+9h1Nht/vB9yr1/5ENCnFtXS5t0pPZxuGN Nx3nxniny6K9Q3MPO1gN9/qGFrRH2qH/d/1JDlc2sZr1tXxoLtYrtg== =/xpG -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Kernel panics even in OSMalloc_noblock() while holding a spinlock
by Terry Lambert 12 Aug '09

12 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi Terry, Yes. You are missing several things... There are actually several possible causes for a panic here: (1) Blocking while holding a spinlock (2) Blocking in interrupt context (3) Unlocking an already unlocked mutex (4) Unlocking an already unlocked spinlock The "preemption_level 1" indicates that it's either 31 or #2. In any case, it's rather easy to change your code like so: // djikstra_foo protected by being a stack variable void *djikstra_foo = OSMalloc(8, tag); // take the lock; don't do anything that would block for a long time lck_spin_lock(mylock); while (true) { // foo protected by the spinlock void* foo = djikstra_foo; } // drop the spinlock as soon as humanly possible lck_spin_unlock(mylock); } -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Aug 11, 2009, at 2:12 PM, Eric Ogren wrote: Thanks for your response, although I'm not sure it really answered my question. More details are inline. On Fri, Aug 7, 2009 at 7:54 PM, Terry Lambert <tlambert(a)apple.com> wrote: On Aug 7, 2009, at 3:50 PM, Eric Ogren wrote: Am I missing something here, or is it unsafe to even call OSMalloc_noblock() while holding a spinlock? If I look at the source code for zalloc, lock/unlock_zone() is always called regardless of the canblock parameter, and that zone lock is indeed a mutex. Seems almost like the canblock parameter just means that the calling thread will not block for a long time (ie will not try to refill or garbage collect the zone if it's full), not that it will never block at all. (A) You're in a tight while/true loop allocating all of kernel memory a tiny bit at a time until you exhaust it, which is known to cause a panic. (B) An infinite loop is too long a time to hold a spinlock; holding a spinlock too long is known to cause a panic Sure, I agree that this test program would ultimately panic the kernel for other reasons but thought it was clear from the backtrace that the panic was occurring from a thread_block() call. More on this below. (C) You're missing your paste of at least the lines above the portion of the backtrace you quoted (which is cut off at frame #2), which would include the actual panic message and frame #1 so that we could see if the panic was related to holding a spinlock or related to you exhausting the zone of zones, or running the wrong version of Parallels or dereferencing a NULL pointer, or some other known cause of panics, or some other unknown cause of panics. I was trying to trim parts of the stack trace just to make the message a little shorter - I didn't think the calls made after panic() were really relevant. I should have included the actual panic message instead of making people try to infer it via the callstack though. The full trace including paniclog looks like this (this is a second run - not 100% sure why the stack is 1 frame shorter this time around) panic(cpu 0 caller 0x001368FD): "thread_invoke: preemption_level 1\n"@/SourceCache/xnu/xnu-1228.15.4/osfmk/kern/sched_prim.c:1478 (gdb) where #0 Debugger (message=0x8001003b <Address 0x8001003b out of bounds>) at /SourceCache/xnu/xnu-1228.15.4/osfmk/i386/AT386/model_dep.c:799 #1 0x0012b4c6 in panic (str=0x1 <Address 0x1 out of bounds>) at / SourceCache/xnu/xnu-1228.15.4/osfmk/kern/debug.c:275 #2 0x001368fd in thread_invoke (self=0x3db0e40, thread=0x54038b8, reason=0) at /SourceCache/xnu/xnu-1228.15.4/osfmk/kern/sched_prim.c: 1477 #3 0x00136e92 in thread_block_reason (continuation=0x1, parameter=0x0, reason=<value temporarily unavailable, due to optimizations>) at /SourceCache/xnu/xnu-1228.15.4/osfmk/kern/ sched_prim.c:1837 #4 0x00136f20 in thread_block (continuation=0x1) at /SourceCache/ xnu/xnu-1228.15.4/osfmk/kern/sched_prim.c:1854 #5 0x001318f1 in lck_mtx_lock_wait (lck=0x28ce084, holder=0x3db3410) at /SourceCache/xnu/xnu-1228.15.4/osfmk/kern/ locks.c:601 #6 0x0019d8c1 in lck_mtx_lock () at pmap.h:176 #7 0x001433b0 in zalloc_canblock (zone=0x28ce07c, canblock=0) at / SourceCache/xnu/xnu-1228.15.4/osfmk/kern/zalloc.c:883 In this case, you already hold the spinlock, your preepmtion level has been raised, and now you are attempting to block taking the zone lock in order to do your allocation, but someone (perhaps even someone blocking) is already sitting on the zone lock holding it, so you fall down to thread_block_reason(), which attempts to reschedule and blows up because of the already raised preemption level on the thread. #8 0x0012fdc2 in kalloc_canblock (size=8, canblock=0) at / SourceCache/xnu/xnu-1228.15.4/osfmk/kern/kalloc.c:289 #9 0x001301c1 in OSMalloc_noblock (size=8, tag=0x723c900) at / SourceCache/xnu/xnu-1228.15.4/osfmk/kern/kalloc.c:303 #10 0x2051c05a in alloc_func (myarg=0x3962b80) at /Users/eogren/ Documents/allocer/allocer.c:26 (D) You have already been told you can use Djikstra's algorithm, in which you speculatively do an allocation before holding the spinlock, and if you use it, fine, mark it as consumed at the point you would have done your allocation, and if you don't, fine, free it after you have dropped your spinlock, no harm no foul, no allocation or free inside a spinlock This was the first time I have ever posted to this mailing list, so I assume you are just referring to other posts in the archive when you say I have been told this. Yes, sorry. Since you referred to the list archives, I had assumed you'd already seen the advice in the historical thread about this. Mike Smith did make it pretty clear, and then Shailesh Jain jumped in and confused things (well, me anyway 8-)). Regardless, I was not trying to argue that calling OSMalloc_noblock() inside of a spinlock is the greatest design-- I was just trying to confirm for my sake and those of others that may browse the archives later that OSMalloc_noblock() actually may indeed block; therefore, since we cannot guarantee that no one else will try to allocate from the kalloc zones at the same time as us, it is not safe to ever call the OSMalloc family while holding a spinlock. As Brendan mentioned in his post, this is important to know when porting code from other platforms. It depends on how big an allocation you are trying to do. If it's small enough it falls into a zone, then you're going to take the zone lock, and you might race it with someone else and shoot yourself in the foot. FreeBSD has a similar zone allocator. I'm pretty sure that other platforms will just block you indefinitely instead of counting down and then panic'ing, so instead of a panic, if your driver exhibited the behaviour ans was using a spinlock, it would just sit there and spin until you freed up the spinlock in another thread or the CPU hit thermal shutdown (and you better hope you have multiple CPUs so whoever is going to let it go is able to make progress, or that they let it go at interrupt context). I think I'm voting on this one being a feature that shows up bugs in code so that you can fix it on all your platforms, rather than a porting issue... void alloc_thread(void *arg) { lck_spin_t* mylock; // XXX This lock should have the same scope as what it's protecting!!! // ... initialize lock This email sent to site_archiver(a)lists.apple.com

1 0

Re: Kernel panics even in OSMalloc_noblock() while holding a spinlock
by Eric Ogren 11 Aug '09

11 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:reply-to:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=pAiKHGLG19kOcd+aGXFNtRPUYOYi4zByvuI66eSrHHo=; b=s7+pryeeuZQedrmorHxcsT60pAK+oaB6ORU424MA81D+z5HypOFPVwIFYdRe2xTuy2 pUZcKsbhRmsG7Hud7xsiE7nHe2F9UhRWCPCptvarpxBBAh8yJE261prjpTVlznB3rHNJ jBDxouNXMh3kzCDIq4LzqaWcUxY7DbXAuQukc= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; b=XrRQj3qYzI204JRlj5SPYd0cA5kZvkzbNgb3+3SBadZpHzsrmakH/9jeZotkt+P0va 5J6voNoQ1xoX+5bKWucRvpJ+oZQQc5ZbJIz0llvygWcB+869eyVS42NEHSaA80jIgkBf thhGm9Agd7AFRMRCZvP26DiS4L/vPZ+nWBR98= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Kernel panics even in OSMalloc_noblock() while holding a spinlock
by Peter Lovell 10 Aug '09

10 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Aug 8, 2009, at 4:26 AM, Terry Lambert wrote: This is a quote for the ages -- I've added it to my list (ROFL) _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… ... hardware people ... are just not that imaginative. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Kernel panics even in OSMalloc_noblock() while holding a spinlock
by Terry Lambert 08 Aug '09

08 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com -- Terry On Aug 7, 2009, at 11:03 PM, Brendan Creane <bcreane(a)gmail.com> wrote: "...no allocation or free inside a spinlock..." That's actually new information -- previous posts (http://lists.apple.com/archives/darwin-kernel/2008/Jan/msg00007.html) do indeed suggest OSMalloc_noblock() as a solution for allocating memory from within a spinlock. My reading of OSMalloc_noblock() is that it may block when obtaining the zone mutex, but it won't wait for any other threads to complete a memory allocation. So it's more like "OSMalloc_fast()." And yes, it may panic inside a spinlock, just like OSMalloc(). For those of us porting existing code from operating systems that do support memory allocation from within lock primitives (Linux, Windows), we need to be aware that Darwin's design precludes this approach and consequently may require a significant rewrite of existing drivers. regards, Brendan Hello there - void alloc_thread(void *arg) { lck_spin_t* mylock; // ... initialize lock lck_spin_lock(mylock); [ ... ] Yes. You are missing several things... (B) An infinite loop is too long a time to hold a spinlock; holding a spinlock too long is known to cause a panic -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/bcreane%40gmail.com This email sent to bcreane(a)gmail.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… That's not really an accurate representation. As long as no one else tries to acquire the spinlock while you hold it, there's no chance of panic; you could hold the thing for years and there would be no panic. You want that post to be true? Then quit reentering on the lock; magically, it's now true. As soon as it's contended, though, the countdown clock starts. Correctly written, scalable code will minimize contention and minimize lock hold times, no matter what OS it's initially written to run on. In case it's not clear from that: locks in parallel systems need to protect data, NOT critical sections of code. The more parallel the hardware becomes, the more important that idea is. Doing a CLI, running a bunch if code, then doing an STI is not the path to the future. Portable code, on the other hand, will not rely on specific types of synchronization primitives; the difference between a spinlock and a mutex is not about what it does as a synchronization primitive, but about what it costs you to do it (so long as you do it fast!). Picking a semantically identical (assuming multiple processors or kernel preemption) primitive based on cost rather than on a considered examination of the problem is likely the wrong thing to do. That's like saying "I'll use vfork() because it's just like fork(), only faster". Yeah, it doesn't copy the address space, but there are other tradeoffs besides that one. If you come into it with your eyes open, it's not going to be hard to do a driver port from any system to any other system; it's not like learning new customs on an alien planet: hardware people (and the laws of physics they have to obey) are just not that imaginative. On Fri, Aug 7, 2009 at 7:54 PM, Terry Lambert<tlambert(a)apple.com> wrote: On Aug 7, 2009, at 3:50 PM, Eric Ogren wrote: I am working on a kernel extension that sometimes attempts to allocate memory while holding a spinlock (lck_spin_t). I've read in several postings to this list that doing so while OSMalloc can cause a kernel panic, and that developers should instead use OSMalloc_noblock() and be prepared to deal with a NULL result. However, I am seeing kernel panics even when calling the noblock variant. The panic is occurring inside zalloc_canblock () at the lock_zone() call, which is indeed trying to block. This problem is very easily reproducible with a simple kext that spawns 2 threads with the following thread routine: while (true) { void* foo = OSMalloc_noblock(8, tag); // tag is a global variable initialized in the start function } } Loading the kext panics the system almost immediately with the following stack, which is the same as the stack as I come across in the occasional panics: #2 0x0012b4c6 in panic (str=0x1 <Address 0x1 out of bounds>) at /SourceCache/xnu/xnu-1228.12.14/osfmk/kern/debug.c:275 #3 0x001368fd in thread_invoke (self=0x43018b8, thread=0x39a7c80, reason=0) at /SourceCache/xnu/xnu-1228.12.14/osfmk/kern/ sched_prim.c:1477 Am I missing something here, or is it unsafe to even call OSMalloc_noblock() while holding a spinlock? If I look at the source code for zalloc, lock/unlock_zone() is always called regardless of the canblock parameter, and that zone lock is indeed a mutex. Seems almost like the canblock parameter just means that the calling thread will not block for a long time (ie will not try to refill or garbage collect the zone if it's full), not that it will never block at all. (A) You're in a tight while/true loop allocating all of kernel memory a tiny bit at a time until you exhaust it, which is known to cause a panic. (C) You're missing your paste of at least the lines above the portion of the backtrace you quoted (which is cut off at frame #2), which would include the actual panic message and frame #1 so that we could see if the panic was related to holding a spinlock or related to you exhausting the zone of zones, or running the wrong version of Parallels or dereferencing a NULL pointer, or some other known cause of panics, or some other unknown cause of panics. (D) You have already been told you can use Djikstra's algorithm, in which you speculatively do an allocation before holding the spinlock, and if you use it, fine, mark it as consumed at the point you would have done your allocation, and if you don't, fine, free it after you have dropped your spinlock, no harm no foul, no allocation or free inside a spinlock This email sent to site_archiver(a)lists.apple.com

1 0

Re: Kernel panics even in OSMalloc_noblock() while holding a spinlock
by Brendan Creane 08 Aug '09

08 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=udBTTDciUEDWGBFMytNg+gx/jVz+JHp/2JNGDcCdwqg=; b=E1JE9y2kerqbvG2LoAuQKum/nrtJMuMX+lH/XVYQrdYkO6FVPGZCEUgJ7elML0d3cN AvnF+fLg1zU522dVlAhuFJ2dV4RvVR81rlRPZSQPNEGNz0syA27IrfBaQx1nFdhY9nej 61TyrQE/gU55QQRays1ShTo8wCYcxRgVe8KdQ= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=McVuwLff1ex59wd5CCsFfc05wLF831cmlQXzbEM6mI0Hbs04MHG1kJHJaUCQ8QXVaq 6RgV2ciWER7q9+vjjIvNGSss58K5Wnxbzl8FyCEnhGzpLagCJWK+d8i/Y4GmihSMYZ2s aQUnsWoOO8JOh265ngvqP3Gx6ThIsNOlyrT4Y= "...no allocation or free inside a spinlock..." That's actually new information -- previous posts (http://lists.apple.com/archives/darwin-kernel/2008/Jan/msg00007.html) do indeed suggest OSMalloc_noblock() as a solution for allocating memory from within a spinlock. My reading of OSMalloc_noblock() is that it may block when obtaining the zone mutex, but it won't wait for any other threads to complete a memory allocation. So it's more like "OSMalloc_fast()." And yes, it may panic inside a spinlock, just like OSMalloc(). For those of us porting existing code from operating systems that do support memory allocation from within lock primitives (Linux, Windows), we need to be aware that Darwin's design precludes this approach and consequently may require a significant rewrite of existing drivers. regards, Brendan On Fri, Aug 7, 2009 at 7:54 PM, Terry Lambert<tlambert(a)apple.com> wrote: > On Aug 7, 2009, at 3:50 PM, Eric Ogren wrote: >> >> Hello there - >> >> I am working on a kernel extension that sometimes attempts to allocate >> memory while holding a spinlock (lck_spin_t). I've read in several postings >> to this list that doing so while OSMalloc can cause a kernel panic, and that >> developers should instead use OSMalloc_noblock() and be prepared to deal >> with a NULL result. However, I am seeing kernel panics even when calling >> the noblock variant. The panic is occurring inside zalloc_canblock() at the >> lock_zone() call, which is indeed trying to block. This problem is very >> easily reproducible with a simple kext that spawns 2 threads with the >> following thread routine: >> >> void alloc_thread(void *arg) { >> lck_spin_t* mylock; >> // ... initialize lock >> >> lck_spin_lock(mylock); >> >> while (true) { >> void* foo = OSMalloc_noblock(8, tag); // tag is a global variable >> initialized in the start function >> } >> } >> >> Loading the kext panics the system almost immediately with the following >> stack, which is the same as the stack as I come across in the occasional >> panics: >> >> #2 0x0012b4c6 in panic (str=0x1 <Address 0x1 out of bounds>) at >> /SourceCache/xnu/xnu-1228.12.14/osfmk/kern/debug.c:275 >> #3 0x001368fd in thread_invoke (self=0x43018b8, thread=0x39a7c80, >> reason=0) at /SourceCache/xnu/xnu-1228.12.14/osfmk/kern/sched_prim.c:1477 > > [ ... ] > > >> Am I missing something here, or is it unsafe to even call >> OSMalloc_noblock() while holding a spinlock? If I look at the source code >> for zalloc, lock/unlock_zone() is always called regardless of the canblock >> parameter, and that zone lock is indeed a mutex. Seems almost like the >> canblock parameter just means that the calling thread will not block for a >> long time (ie will not try to refill or garbage collect the zone if it's >> full), not that it will never block at all. > > > Yes. You are missing several things... > > (A) You're in a tight while/true loop allocating all of kernel memory a tiny > bit at a time until you exhaust it, which is known to cause a panic. > > (B) An infinite loop is too long a time to hold a spinlock; holding a > spinlock too long is known to cause a panic > > (C) You're missing your paste of at least the lines above the portion of the > backtrace you quoted (which is cut off at frame #2), which would include the > actual panic message and frame #1 so that we could see if the panic was > related to holding a spinlock or related to you exhausting the zone of > zones, or running the wrong version of Parallels or dereferencing a NULL > pointer, or some other known cause of panics, or some other unknown cause of > panics. > > (D) You have already been told you can use Djikstra's algorithm, in which > you speculatively do an allocation before holding the spinlock, and if you > use it, fine, mark it as consumed at the point you would have done your > allocation, and if you don't, fine, free it after you have dropped your > spinlock, no harm no foul, no allocation or free inside a spinlock > > -- Terry > > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/darwin-kernel/bcreane%40gmail.com > > This email sent to bcreane(a)gmail.com > _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0