December 2009 - Darwin-kernel

Background Thread and Installer
by Duane Murphy 29 Dec '09

29 Dec '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 We have a problem with our kext where our background thread in the kernel is not shutting down during a system update installation. Our kext that is a subclass of IOBlockStorageDriver. The kext runs a background thread directly processing raw sectors from the disk. During a system update installation, all user processes quit except for the installation process. Our kernel thread keeps running in the background just fine. The problem is that the kext is never notified of the need to shutdown the system, so the system appears to just hang as our thread continues to run. We see this most reliably when updating from 10.6 to 10.6.2 (Combo updater). We also see a variation of this when performing additional system software updates on 10.6.2. However, in this case, I believe a timeout has been added (possibly to alleviate this problem). After about 2 minutes following the completion of the installer, a system shutdown notice is given to the kext (via our registerPrioritySleepWakeInterest callback function). But even in this case, the process is not allowed to complete properly and the system never completes shutting down. The system ends up in an odd state where even an NMI won't register, but the system is not shutdown. My suspicion is that we must be holding some kind of semaphore-like object that is preventing the system from shutting down, but I don't know what it is. Or the system is just waiting for threads to go idle, in which case I need to work out what I should be watching in order to idle the thread. If I manually close the thread (with the help of gdb), the system shuts down normally. The thread waiting for the system to shutdown looks like: thread_invoke thread_block_reason thread_block lck_rw_sleep vm_fault_page vm_object_update memory_object_lock_request ubc_setsize hfs_vnop_inactive VNOP_INACTIVE vclean vnode_reclaim_internal vflush dounmount vfs_unmountall boot reboot unix_syscall lo_unix_syscall What is this thread waiting for and how can I know? What event am I missing? How can I know to close this thread to allow the system to shutdown properly? Note, I'd like to keep the disk process running in the background as long as possible, until the installation process is complete. Thanks for your help, ...Duane -----BEGIN PGP SIGNATURE----- Version: PGP Desktop 9.10.0 (Build 500) Charset: US-ASCII wsBVAwUBSzlJRUrg9acQ4r2CAQiLEQf+KAt4SQhuhfrKm7MdxqaG1TTLNSIr5sxb +lTE75maLxATkVq+DCs0QGOrrDcyflC8ctQoUa4askmdAbtBTU/V4/HOKYw89ctI bQpfV26SJDXIv1ke2EY+J+1JTxukXG21jP7ZJVQIZ1A+Nrt4A+Pz/p1IhA7keuB9 4X1w7MLDRsfI0dm8iv9L3+DfEKrqY9nNkrUK9BnoXuPfXdIDyJbRGHh8lW+NoOwr I8oGyQywBf1y7EtCoScoKhwJDHxGDwfKV93YQBBqke3HJ4VH0e+V63dnkKQ7n+gf k4rLIwvUSXKnGECKc3KTU0fW8gTj6PU4jzz2w6dvAvO4HAgX52xN0g== =hfw4 -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: darwin8 thread cancellation?
by Per Mildner 21 Dec '09

21 Dec '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Dec 21, 2009, at 5:29 AM, David Fang wrote: Hi all, Regards, Per Mildner Per.Mildner(a)sics.se _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I've been testing out GNU guile-1.9.6 (alpha) for a few days, preparing an experimental fink package, and ran into a test failure in their test suite which seeks explanation. On OS X 10.4 (powerpc- darwin8), their tests on threads hang indefinitely in the section on thread-cancellation. The same (and all) tests pass on 10.6 (i686- darwin10). How is thread cancellation supported in darwin8 (if at all) differently than in darwin10? I'm looking for any hints on where to start debugging, if this is something that can be fixed. Thanks in advance! Also note that 10.5 has more cancellation points but still does not implement the SUSv3 mandated cancellation points in I/O (I reported this as Apple Bug ID# 5610812). Thus 10.5 does not conform to SUSv3, but you do not want to bring that up on this list :-) Apple has not informed me whether this bug is fixed in 10.6 or not but my limited tests indicate that thread cancellation of threads doing blocking I/O does work in 10.6 which is great news. This email sent to site_archiver(a)lists.apple.com

1 0

Re: darwin8 thread cancellation?
by Jonas Maebe 21 Dec '09

21 Dec '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On 21 Dec 2009, at 05:29, David Fang wrote: Jonas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I've been testing out GNU guile-1.9.6 (alpha) for a few days, preparing an experimental fink package, and ran into a test failure in their test suite which seeks explanation. On OS X 10.4 (powerpc- darwin8), their tests on threads hang indefinitely in the section on thread-cancellation. The same (and all) tests pass on 10.6 (i686- darwin10). How is thread cancellation supported in darwin8 (if at all) differently than in darwin10? I'm looking for any hints on where to start debugging, if this is something that can be fixed. I guess that Mac OS X 10.4 simply supported fewer cancellation points. IIRC, the only one it supported was pthread_testcancel. See http://lists.apple.com/archives/Darwin-kernel/2004/Jan/msg00032.html This email sent to site_archiver(a)lists.apple.com

1 0

darwin8 thread cancellation?
by David Fang 21 Dec '09

21 Dec '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi all, Fang David Fang http://www.csl.cornell.edu/~fang/ http://www.achronix.com/ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I've been testing out GNU guile-1.9.6 (alpha) for a few days, preparing an experimental fink package, and ran into a test failure in their test suite which seeks explanation. On OS X 10.4 (powerpc-darwin8), their tests on threads hang indefinitely in the section on thread-cancellation. The same (and all) tests pass on 10.6 (i686-darwin10). How is thread cancellation supported in darwin8 (if at all) differently than in darwin10? I'm looking for any hints on where to start debugging, if this is something that can be fixed. Thanks in advance! This email sent to site_archiver(a)lists.apple.com

1 0

Re: Building 10.6 kernel?
by William Kucharski 18 Dec '09

18 Dec '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com User-agent: Thunderbird 2.0.0.23 (X11/20090910) Andreas Fink wrote: Well let's be fair here. William Kucharski _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… AppleProfileFamily needs some secret xnu bits that aren't provided in the open source releases, so when it loads it fails to export a necessary IOResource, which various other dependent kexts block waiting for. so what's open source worth if its not the same kernel than stock MacOS X. Do we now have to go and reverse engineer something which is supposed to be open source to start something apple is breaking? Apple shouldn't have dependencies on closed code in what they open source, but at the same time they at least deserve credit for open sourcing everything they CAN release. So if "A" relies on "B" which is closed source, Apple could either release the source for "A" or keep "A" closed as well. Apple's doing the former. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Building 10.6 kernel?
by Andreas Fink 18 Dec '09

18 Dec '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On 18.12.2009, at 06:26, Wade Tregaskis wrote: >> Thanks for all the help. The advice I received worked great. I no longer see >> the 10 minute delay at system startup. Below is a summary of the >> steps I took to build and run a 10.6.2 kernel. However, I would still like to >> know why the AppleProfileFamily kext needs to be disabled. Could someone shed >> some light on exactly how the AppleProfileFamily kext is dependent upon the >> specific kernel that it ships with? > > AppleProfileFamily needs some secret xnu bits that aren't provided in the open source releases, so when it loads it fails to export a necessary IOResource, which various other dependent kexts block waiting for. so what's open source worth if its not the same kernel than stock MacOS X. Do we now have to go and reverse engineer something which is supposed to be open source to start something apple is breaking? > On the upside, at least it was thought to put a timeout on the waiting, even though this problem was never envisioned. :) > > AppleProfileFamily (and friends) are required for Instruments' Time Profiler, but nothing else currently. Since the necessary parts of xnu aren't there, it wouldn't work in any case, so there's no further harm in disabling it as was previously suggested. except if you use Instruments... > It's unlikely that the necessary xnu bits will be open sourced, though I encourage you to file bug reports if you have a need for Time Profiler (or Shark, which is similarly afflicted) on an open source xnu. this turns open source ad absurdum.. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

OT: Semaphore Interview Question
by Abdulla Kamar 18 Dec '09

18 Dec '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:from:date:message-id :subject:to:content-type; bh=MVwSP9O54lIW4n9PTqqvCe5eB2j19ZD3gSQOFVdnNPo=; b=SEmyX8xw8+lWldLhUwONVCrzruz7x0XMeBdMPl3p4eqYmz+XsnBSnwStwlnim91guR vWRP5ZVLwWuQ+eH/iwhjLORpbi5VZz7SRuHp70zEwCgd6A1fuaCPe81lvYyFi26dTijM jRxm8EewF1Sy7BvjL4Y8/w2Og86kxl+XR/Kmk= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:from:date:message-id:subject:to:content-type; b=ndnvxSCqNojBlIHu5rv6gDayN+63/8yaAjYeedDIgVrh1enO7KB6vlOoDsgSgGGfWe RYDI0O4Sc9ytaU9QRP4rdnpUhb265xayW+bsfImyYS8lfsSEhifQkV8To4IxEcG0c6ba 08CH6Y0JO/thC3m4jK3S7MgE2LbRvjIT3oRBE= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Building 10.6 kernel?
by Wade Tregaskis 18 Dec '09

18 Dec '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com > Thanks for all the help. The advice I received worked great. I no longer see > the 10 minute delay at system startup. Below is a summary of the > steps I took to build and run a 10.6.2 kernel. However, I would still like to > know why the AppleProfileFamily kext needs to be disabled. Could someone shed > some light on exactly how the AppleProfileFamily kext is dependent upon the > specific kernel that it ships with? AppleProfileFamily needs some secret xnu bits that aren't provided in the open source releases, so when it loads it fails to export a necessary IOResource, which various other dependent kexts block waiting for. On the upside, at least it was thought to put a timeout on the waiting, even though this problem was never envisioned. :) AppleProfileFamily (and friends) are required for Instruments' Time Profiler, but nothing else currently. Since the necessary parts of xnu aren't there, it wouldn't work in any case, so there's no further harm in disabling it as was previously suggested. It's unlikely that the necessary xnu bits will be open sourced, though I encourage you to file bug reports if you have a need for Time Profiler (or Shark, which is similarly afflicted) on an open source xnu. Wade _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Serious Interruptive Pauses
by Brian Bergstrand 17 Dec '09

17 Dec '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Dec 17, 2009, at 6:57 PM, George Plymale wrote: > Well, the plot thickens perhaps. I came back to my workstation this morning and sometime over the last couple of hours it panicked: > ... > loaded kexts: > com.microsoft.driver.MicrosoftMouseBluetooth 7.1 > com.microsoft.driver.MicrosoftKeyboardBluetooth 7.1 > com.microsoft.driver.MicrosoftKeyboard 7.1 > com.vmware.kext.vmnet 3.0.0 > com.vmware.kext.vmioplug 3.0.0 > com.vmware.kext.vmci 3.0.0 > com.vmware.kext.vmx86 3.0.0 > com.parallels.kext.prl_vnic 5.0 9220.531002 > com.parallels.kext.prl_netbridge 5.0 9220.531002 > xxx.qnation.PeerGuardian 1.1.10 > com.parallels.kext.prl_usb_connect 5.0 9220.531002 > com.parallels.kext.prl_hid_hook 5.0 9220.531002 > com.parallels.kext.prl_hypervisor 5.0 9220.531002 > com.Cycling74.driver.Soundflower 1.4.4 > com.DataRobotics.driver.iSCSIInitiator 1.4.20 > com.microsoft.driver.MicrosoftMouseUSB 7.1 > com.microsoft.driver.MicrosoftMouse 7.1 > com.TrustedData.driver.VendorSpecificType00 1.3.3 > at.obdev.nke.LittleSnitch 2.2.05 > You have a ton of 3rd party kexts loaded. Most likely one of them is the culprit. You should unload all of them and attempt to reproduce the issue. If you can't, then start loading them back one-by-one until you do reproduce the issue and the culprit is identified. p.s. I'd also advise against running two virtualization programs at once, they are going to compete for a lot of resources. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to add mbuf to the head of mbuf chain
by Adi Masputra 17 Dec '09

17 Dec '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com The assertion occurred because you replaced the contents of pkthdr (which is private) with something else that doesn't belong with that mbuf; don't use mbuf_pkthdr_setheader(). I'm not quite clear on what you're trying to do, but instead, if you want to prepend data, use mbuf_prepend(). If you want to associate private data to the mbuf, use mbuf_tag_*. If you really want to change the linkage of mbufs, use mbuf_setnext(). For the latter, you'd have to copy over the contents of the head mbuf to another mbuf, set the mbuf length to 0, and point the next pointer to your mbuf which then points to the mbuf containing the original data. Adi On Dec 17, 2009, at 9:15 AM, Dmitriy Arekhta wrote: > Hi List, > > I'm working with interface filter and I should add to the output network packets custom header. In my ctl_send routine I have pointer to the source mbuf, so I do the following to chain custom mbuf to front: > > UInt32 srcMbufDataLen = mbuf_chain_size(m); // len of current buffer > > UInt32 dstMbufDataLen = srcMbufDataLen; > mbuf_t dstMbuf = NULL; // mbuf that will be passed to dlil > mbuf_t mbufToSend = m; > unsigned int dstMbufChunks = 1; > > UInt32 srcMbufOffset = 0; > errno_t result = KERN_SUCCESS; > > KTHPAdapter *adapter = static_cast<KTHPAdapter *>(userdata); > > > if (adapter->fFilter.IsThpModeEnabled) > { > dstMbufDataLen += sizeof(KTHPPacketHeader); > > // Let's try to allocate new mbuf > result = mbuf_allocpacket(MBUF_DONTWAIT, // blocking or non-blocking > sizeof(KTHPPacketHeader), // entire packet len > &dstMbufChunks, // total packet chunks > &dstMbuf > ); > > if (result != KERN_SUCCESS || !dstMbuf || !mbuf_data(dstMbuf)) > { > ALERT(dstMbufDataLen, result, "Couldn't allocate packet to send, dstMbufDataLen, result"); > mbuf_freem(m); > > return result; > } > > mbuf_setlen(dstMbuf, sizeof(KTHPPacketHeader)); > > mbufToSend = dstMbuf; > > mbuf_setnext(mbufToSend, m); > > // Update an old packets > for(mbuf_t c = m; c != NULL; c = mbuf_next(c)) > { > mbuf_pkthdr_setheader(c, mbuf_data(mbufToSend)); > } > mbuf_pkthdr_setlen(m, dstMbufDataLen); > > // Update the new packet > mbuf_setflags(mbufToSend, mbuf_flags(m)); > mbuf_settype(mbufToSend, mbuf_type(m)); > > mbuf_pkthdr_setheader(mbufToSend, 0); > mbuf_pkthdr_setlen(mbufToSend, dstMbufDataLen); > > // Clear status fiels of the old mbuf > mbuf_setflags(m, 0); > > } > > When I try to send this packet to the network interface I got the kernel panic in m_freem_list(struct mbuf *m) > > panic(cpu 0 caller 0x0038DA21): assertion failed: m->m_ext.ext_free == m_16kfree, file: /SourceCache/xnu/xnu-1228.15.4/bsd/kern/uipc_mbuf.c, line: 3639 > > What do I do wrong? Any advice? > Thanks :) > > Dmitri > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/darwin-kernel/adi%40apple.com > > This email sent to adi(a)apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0