December 2005 - Darwin-kernel

Re: Race condition in libkern's OSAtomicDequeue()
by Kevin Van Vechten 06 Dec '05

06 Dec '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com - Kevin On Dec 6, 2005, at 5:51 AM, Colin Hirsch wrote: Hi, But first the issue: In xnu-792.6.22/libkern/gen/OSAtomicOperations.c the following code is used for the dequeue. I believe the problem to be that the value in newListHead is not necessarily current by the time the atomic CAS is executed. This is a pretty contrived case, as it requires one thread to be preempted between the assignment to newListHead and the CAS, and then two (or more) elements must be dequeued and at least the first dequeued element be put back again [last] so that the CAS succeeds. void * OSDequeueAtomic(void ** inList, SInt32 inOffset) { void * oldListHead; void * newListHead; do { oldListHead = *inList; if (oldListHead == NULL) { break; } return oldListHead; } In practice, my test program http://www.colin-hirsch.net/source/list_broken.cc http://www.colin-hirsch.net/source/list_fixed.cc Regards, Colin _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… You might want to report this to <http://developer.apple.com/ bugreporter/> in addition to posting to the list. There are some benefits to you for doing this. These lists are read by Apple engineers on a voluntary basis, whereas responding to bug reports is a bit more compulsory. And you'll be able to track its progress. playing with all kinds of SMB-safe atomic operations I stumbled over the OSAtomicDequeue() and ...Enqueue() operations in the kernel source code. The reason I was looking there was that my first singly-linked-list kept crashing immediately (in a test program that does nothing but enqueue()/ dequeue() in 3 threads on my Quad G5). However the libkern function has the same problem, as explained below and as can be verified easily with the source linked below. newListHead = *(void **) (((char *) oldListHead) + inOffset); } while (! OSCompareAndSwap((UInt32)oldListHead, (UInt32)newListHead, (UInt32 *)inList)); which uses the same list operations as libkern, crashes immediately, whereas the "fixed" version has been running for several minutes now, so it might actually be correct ;-) The crux is to temporarily take ownership of the complete list during dequeue() to prevent other CPUs from doing the aforementioned sequence of dequeues()s and enqueue()s, and then giving the rest of the list back (i.e. the list minus the dequeued head). I would be grateful if somebody (at Apple?) could double-check my analysis and the fixed version and, if applicable, crate a patch for libkern... (My fixed version is not fit for general purpose use since it does not detect an empty list.) PS: Yes, the commented "isync" in the fixed version is at the wrong place. This email sent to site_archiver(a)lists.apple.com

1 0

Race condition in libkern's OSAtomicDequeue()
by Colin Hirsch 06 Dec '05

06 Dec '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com User-agent: Mutt/1.5.9i Hi, playing with all kinds of SMB-safe atomic operations I stumbled over the OSAtomicDequeue() and ...Enqueue() operations in the kernel source code. The reason I was looking there was that my first singly-linked-list kept crashing immediately (in a test program that does nothing but enqueue()/ dequeue() in 3 threads on my Quad G5). However the libkern function has the same problem, as explained below and as can be verified easily with the source linked below. But first the issue: In xnu-792.6.22/libkern/gen/OSAtomicOperations.c the following code is used for the dequeue. I believe the problem to be that the value in newListHead is not necessarily current by the time the atomic CAS is executed. This is a pretty contrived case, as it requires one thread to be preempted between the assignment to newListHead and the CAS, and then two (or more) elements must be dequeued and at least the first dequeued element be put back again [last] so that the CAS succeeds. void * OSDequeueAtomic(void ** inList, SInt32 inOffset) { void * oldListHead; void * newListHead; do { oldListHead = *inList; if (oldListHead == NULL) { break; } newListHead = *(void **) (((char *) oldListHead) + inOffset); } while (! OSCompareAndSwap((UInt32)oldListHead, (UInt32)newListHead, (UInt32 *)inList)); return oldListHead; } In practice, my test program http://www.colin-hirsch.net/source/list_broken.cc which uses the same list operations as libkern, crashes immediately, whereas the "fixed" version http://www.colin-hirsch.net/source/list_fixed.cc has been running for several minutes now, so it might actually be correct ;-) The crux is to temporarily take ownership of the complete list during dequeue() to prevent other CPUs from doing the aforementioned sequence of dequeues()s and enqueue()s, and then giving the rest of the list back (i.e. the list minus the dequeued head). I would be grateful if somebody (at Apple?) could double-check my analysis and the fixed version and, if applicable, crate a patch for libkern... (My fixed version is not fit for general purpose use since it does not detect an empty list.) Regards, Colin PS: Yes, the commented "isync" in the fixed version is at the wrong place. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Where is the filename?
by Pavan Chebbi 06 Dec '05

06 Dec '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=fQW+1yYp8nNlCz8MF9ommOe11wuavKbbqht8Cot1QnYIPXsXAfNfmVma+IwmGFoYm0nUpO/maiXtwVTfbERUV5LOZhisZUThJg//zAv7D0T4++Zny3JtdbwlbdDz+kD0s606EQjDeZGhgWGWDelhRbxL3ypDheolcAme1eiOZm0= ; _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: socket creation synchronization
by Josh Graessley 06 Dec '05

06 Dec '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com -josh On Dec 5, 2005, at 3:53 PM, Christopher Vitale wrote: Hello, I'm calling this: socket_t so; sock_socket(PF_SYSTEM, SOCK_DGRAM, SYSPROTO_CONTROL, NULL, NULL, &so); This should be fine. It works almost all the time. ;-) In my stack trace shows sock_socket calling socreate, which calls pffindproto. struct protosw * pffindproto(family, protocol, type) int family, protocol, type; { register struct protosw *pr; lck_mtx_assert(domain_proto_mtx, LCK_MTX_ASSERT_NOTOWNED); lck_mtx_lock(domain_proto_mtx); pr = pffindproto_locked(family, protocol, type); lck_mtx_unlock(domain_proto_mtx); return (pr); } Thanks, _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com The assert is asserting that the mtx is not locked by the current thread. One of two things is happening here. Either your code is calling this from code that is already holding the lock or there is an edge case somewhere where we take the lock but don't drop it. If you can get the backtrace from the panic, that will help determine if you're being called in a situation where the domain_proto_mtx lock is already held. I die on the lck_mtx_assert, xnu-792.6.22/bsd/kern/uipc_domain.c: 367. I'm still in the generic socket handling code at this point. The fact that I'm creating a SYSPROTO_CONTROL socket shouldn't matter. I think a userland socket may be holding the domain_proto_mtx lock. A remote employee is the only one that cause the crash consistently, I've been relying on his panic.log. A two-machine debug remains elusive. vitale smime.p7s

1 0

Re: socket creation synchronization
by Christopher Vitale 05 Dec '05

05 Dec '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thread-index: AcX54xG2UG/762XWEdqu1wANk3fXlAAFAmUE Thread-topic: socket creation synchronization User-agent: Microsoft-Entourage/11.2.1.051004 Hello, I'm calling this: socket_t so; sock_socket(PF_SYSTEM, SOCK_DGRAM, SYSPROTO_CONTROL, NULL, NULL, &so); This should be fine. It works almost all the time. ;-) In my stack trace shows sock_socket calling socreate, which calls pffindproto. struct protosw * pffindproto(family, protocol, type) int family, protocol, type; { register struct protosw *pr; lck_mtx_assert(domain_proto_mtx, LCK_MTX_ASSERT_NOTOWNED); lck_mtx_lock(domain_proto_mtx); pr = pffindproto_locked(family, protocol, type); lck_mtx_unlock(domain_proto_mtx); return (pr); } I die on the lck_mtx_assert, xnu-792.6.22/bsd/kern/uipc_domain.c:367. I'm still in the generic socket handling code at this point. The fact that I'm creating a SYSPROTO_CONTROL socket shouldn't matter. I think a userland socket may be holding the domain_proto_mtx lock. A remote employee is the only one that cause the crash consistently, I've been relying on his panic.log. A two-machine debug remains elusive. Thanks, vitale Subject: Re: socket creation synchronization From: Josh Graessley <email@hidden> Date: Mon, 5 Dec 2005 15:15:52 -0800 Delivered-to: email@hidden Delivered-to: email@hidden What type of socket are you creating? What gets passed in so sock_socket for domain, type and protocol? Adding to the list of protocols by registering a new protosw or domain structure is not supported. The locking model has not been documented. -josh On 12/5/05 4:30 PM, "Christopher Vitale" <cvitale(a)trustednetworktech.com> wrote: > Hello List, > > I have a question about socket creation in 10.4.3. > > You can arrive at socreate by calling the kpi sock_socket, or from the > userland syscall socket. Regardless, socreate calls pffindproto or > pffindtype. These domain functions both assert that nobody else is holding a > certain lock. How is this guaranteed? There isn't even a funnel associated > with the socket syscall in syscalls.master. I looked around the source for a > while but nothing jumped out at me. > > I'm interested because I've got a kext that creates a connection via > sock_socket. When I call sock_socket while other (userland) sockets are > being created I'll occasionally die a horrible death on the pffindproto lock > assert. I'll only create a single connection for the life of the kext. Does > socket creation in the kernel bypass synchronization at a higher level? > > Also, the socket I create is a system socket to another kext. Is this an > appropriate way for kexts to share information? I take the necessary > precautions like not allowing the server kext to unload while connected, > forcing the client kext to disconnect when unloaded, etc. > > Thanks, > > vitale > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/darwin-kernel/cvitale%40trustednetwo… > ech.com > > This email sent to cvitale(a)trustednetworktech.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: socket creation synchronization
by Josh Graessley 05 Dec '05

05 Dec '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com -josh On Dec 5, 2005, at 1:30 PM, Christopher Vitale wrote: Hello List, I have a question about socket creation in 10.4.3. Thanks, vitale _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/jgraessley%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com What type of socket are you creating? What gets passed in so sock_socket for domain, type and protocol? Adding to the list of protocols by registering a new protosw or domain structure is not supported. The locking model has not been documented. You can arrive at socreate by calling the kpi sock_socket, or from the userland syscall socket. Regardless, socreate calls pffindproto or pffindtype. These domain functions both assert that nobody else is holding a certain lock. How is this guaranteed? There isn't even a funnel associated with the socket syscall in syscalls.master. I looked around the source for a while but nothing jumped out at me. I'm interested because I've got a kext that creates a connection via sock_socket. When I call sock_socket while other (userland) sockets are being created I'll occasionally die a horrible death on the pffindproto lock assert. I'll only create a single connection for the life of the kext. Does socket creation in the kernel bypass synchronization at a higher level? Also, the socket I create is a system socket to another kext. Is this an appropriate way for kexts to share information? I take the necessary precautions like not allowing the server kext to unload while connected, forcing the client kext to disconnect when unloaded, etc. This email sent to jgraessley(a)apple.com smime.p7s

1 0

socket creation synchronization
by Christopher Vitale 05 Dec '05

05 Dec '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thread-index: AcX54xG2UG/762XWEdqu1wANk3fXlA== Thread-topic: socket creation synchronization User-agent: Microsoft-Entourage/11.2.1.051004 Hello List, I have a question about socket creation in 10.4.3. You can arrive at socreate by calling the kpi sock_socket, or from the userland syscall socket. Regardless, socreate calls pffindproto or pffindtype. These domain functions both assert that nobody else is holding a certain lock. How is this guaranteed? There isn't even a funnel associated with the socket syscall in syscalls.master. I looked around the source for a while but nothing jumped out at me. I'm interested because I've got a kext that creates a connection via sock_socket. When I call sock_socket while other (userland) sockets are being created I'll occasionally die a horrible death on the pffindproto lock assert. I'll only create a single connection for the life of the kext. Does socket creation in the kernel bypass synchronization at a higher level? Also, the socket I create is a system socket to another kext. Is this an appropriate way for kexts to share information? I take the necessary precautions like not allowing the server kext to unload while connected, forcing the client kext to disconnect when unloaded, etc. Thanks, vitale _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Kern-control and enqueuing data
by Josh Graessley 05 Dec '05

05 Dec '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com smime.p7s

1 0

Re: Kern-control and enqueuing data
by Josh Graessley 05 Dec '05

05 Dec '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com smime.p7s

1 0

Kern-control and enqueuing data
by Sudarshan S 05 Dec '05

05 Dec '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=cgHgLgxaz1TuQ3UckUkpg4+Zca2ZCdC2t+xXQA7YU4n0in26jHkYM5j6zw/UoBcHgjMHZCPTSbUFE8+2i1xZ1xFilKrxTrchRDqA1gU1SY8ktRzAf0adF3/mU/pGl92doE4pqVLlHN/a2mHJdPf6zlcXbKVYLf9DLdZaBVIjzOU= ; _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0