August 2011 - Darwin-kernel

Re: cpu_number()
by Shantonu Sen 19 Aug '11

19 Aug '11

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: cpu_number()
by Prasad Lingutla 19 Aug '11

19 Aug '11

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com > > I know, I could take the hit and use a > spinlock for those buffers, but depending on the configuration Lustre > can log a lot of data and doing that could really hurt performance Instead of taking a spinlock, can you consider using atomic increment(locked cmpxchg) to update the index into the buffer. This worked very well for me in the past for similar requirements. The thread writing at the end of the buffer and wraping around may have to do atomic increment twice depending on your implementation. You don't even need to disable preemption(assuming that size of data written by all threads simultaneously to this buffer is not greater than size of the buffer). Thread may still need to loop around while atomically incrementing the index, but I don't think this will be such a performance hit compared to disabling preemption on a busy system. -Prasad _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

RE: cpu_number()
by David Lee 19 Aug '11

19 Aug '11

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thread-index: AcxeMx8k1qIohLZZThiBki0IIsn6BgAVbZCs Thread-topic: cpu_number() _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: cpu_number()
by Ken Hornstein 19 Aug '11

19 Aug '11

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com >What are you really trying to do, and why do you think getting the CPU you >were scheduled on a nanosecond ago while be correct a nanosecond from now? Well, _since_ you asked ... One of my off-again, on again projects at work is porting Lustre (a high-performance network filesystem that right now only runs on Linux) to OS X. I've made some reasonable progress (it works, but poorly), but there are some things that are hard to make work as intended. One of the things Lustre does is log data into buffers for later debugging. To get rid of lock contention issues during logging what Lustre does is create a series of buffers for each CPU, and when you log something you log it into the buffer corresponding to your CPU. Now you will no doubt ask exactly how do you guarantee that you don't end up logging to the "wrong" CPU buffer? Well, on Linux as part of getting the CPU number you call the function preempt_disable(), which prevents your thread from getting preempted and pins it to the current CPU. When you're done logging your data you call preempt_enable(). My current solution is to simply return a CPU number of 0 for everything and the debug log gets corrupted. Right now I don't use it so it's not a huge deal, but it would be useful at some point to get it working. Now I think this is a reasonable thing to do in this particular case; it doesn't seem unusual to me. But it's not possible with the currently exported KAPIs. Of course there exists ml_preempt_disable() and cpu_number() in the internal kernel API; if those were exported, it would be exactly what I need. If you've got an alternate suggestion on how I could accomplish this, hey, I'd be glad to hear it. I know, I could take the hit and use a spinlock for those buffers, but depending on the configuration Lustre can log a lot of data and doing that could really hurt performance (I'll be the first to admit that the current client has lousy performance, but it's a work in progress!). Another thing to consider is that the eventual goal is to submit this to the open-source Lustre source, and the more changes I have to make to the Lustre sources, the harder the integration will be. That's not to say that Apple should implement every crazy Linux kernel API out there to make my life easier, but it does mean there's a limit as to how much I can change the Lustre code and still meet my long-term goals. --Ken _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: How can I get notified that a kernel thread has been terminated ?
by Ken Hornstein 19 Aug '11

19 Aug '11

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com >I have tried to use a mutex (which is what an IOLock is as far as I can see) >and no luck. It sometimes works, sometimes it doesn't... It doesn't look >like it is the good way to detect a thread termination. While I agree with you about that, I think that perhaps you've got another problem. The filesystem I ported from Linux created what I would charitably call an "assload" of threads. I used mutexes to wait for the threads to finish up before the module was unloaded. I can unload my module just fine. And I don't hear a bunch of people complaining about this problem; if it was a serious problem then you would think it would come up a lot (or at least it would come up at least _once_ by someone else). >From what I remember you never actually _verified_ that this was the actual problem with a debugger; it was just your best guess. You also never posted a crash log or anything else. I'm willing to believe that your module is different enough that it might be triggering something that doesn't come up for most people, but I think the onus is on you at this point to prove that this is really the core issue (if my memory is wrong, then my apologies). And you know what? I took a longer look at this (using the kernel source that anyone can download); it turns out that I was slightly wrong. While there isn't an exported function that waits for a thread to exit (there is an internal one called thread_wait()), you have a possible option. If you look at the places where thread_wait() is used, you've got thread_terminate() and thread_suspend(). Unfortunately, you can't use thread_terminate() on kernel threads other than yourself, so that leaves thread_suspend(). If you try to suspend a thread that has stopped, thread_suspend() will return KERN_TERMINATED. You don't actually want the thread to be suspended, so what you could do is a loop where you do a thread_suspend() and a thread_resume() until thread_suspend() (or thread_resume()) returns KERN_TERMINATED. I would caution that I'm not 100% sure this will work properly; this depends on a thread being marked as not active, and I haven't convinced myself that happens when a thread function simply exits. But that doesn't work, then I _think_ this should work: In your waitee thread: - Set your mutex to indicate that you're done with whatever you need to be done with. - Call thread_terminate() on yourself. In your waiter thread: - Wait for the above mutex - Do the aforementioned thread_suspend()/thread_resume() loop until you get a KERN_TERMINATED return code (or an error). --Ken _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: How can I get notified that a kernel thread has been terminated ?
by Nicolas Connoud 19 Aug '11

19 Aug '11

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=r3ZkYK+4NJExvBF44b44swgmTgZsRIuQTAPNN0KyYN4=; b=XJhwW4iz7mj2Unq7hP6AZh/0sMEUzUeaP5xEqwrRi6majFQqcURNDvcRaP765mNb94 KBbpf/9x/5ZYGKdHFmH0+DWjdNg38P11iuU61DH6dSBDs19zFVYbTsOJyKf8OijMKR+b 9DFMIivZBGaSwfb8n472v+Mmk/n8kjDVO6GtE= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: cpu_number()
by Shantonu Sen 19 Aug '11

19 Aug '11

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

RE: cpu_number()
by Jiang, Yunhong 19 Aug '11

19 Aug '11

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thread-index: AcxeLCwqvU50VFr4TuS0Dzp0DXIrLQABAjQg Thread-topic: cpu_number() _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

cpu_number()
by David Lee 19 Aug '11

19 Aug '11

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thread-index: AcxeLCwqvU50VFr4TuS0Dzp0DXIrLQ== Thread-topic: cpu_number() _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

gdb error: Premature end of file reading symbol table
by mogambo 17 Aug '11

17 Aug '11

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=jCx9A2fCu2DfSUaTI/A2XtUe2uSAG9M8OCK3Un/QOQE=; b=hzJCibazyhtWwwTjwddt4u+ninCQKIap3k/k1mFUe40y3j+RKhZI9+lZtqxIDn/BB1 beMAChYPnIgRYd8Ac1GZqP8DMZcDhfy5pALQBnjQ6gzWY9e5vqzHgOrzqpVnwf2WPjdh YjLsNOndnkxsLCc//lXKgRAYGOCDoSoZlfdGM= Hi, While on the subject of kext debugging with gdb on 10.7, I have a related problem. When I try to debug a panic on 10.7, I see the error when I try to load the symbol file. (gdb) add-symbol-file ~/fsd/symdir/com.omneon.kext.omfs.sym add symbol table from file "~/fsd/symdir/com.omneon.kext.omfs.sym"? (y or n) y Reading symbols from ~/fsd/symdir/com.omneon.kext.omfs.sym...Premature end of file reading symbol table The symbol file is generated with the following command: sudo kextutil -arch x86_64 -s ~/fsd/symdir ~/fsd/omfs.kext The backtrace can not show any frames from my kernel extension. Anyone know about this? Thanks, -m. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0