December 2006 - Darwin-kernel

Re: Re: Threads within the kernel
by Quinn 13 Dec '06

13 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com So, how do I go about starting a low priority thread? Is there another bsd_timeout() like function? o Mach exports kernel_thread_start o I/O Kit exports IOCreateThread S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Relations, Developer Technical Support, Core OS/Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 9:53 -0500 12/12/06, Curtis Jones wrote: I'm not sure what the /best/ answer to this is. However, an answer that will work (probably :-) is to start a new kernel thread and then explicitly set its priority using thread_policy_set. You can start kernel threads in a variety of ways. Which you should use depends on the environment you're working in. If you're knee deep in I/O Kit, IOCreateThread would be best. If you're just using BSD, the Mach routine would be best. Be careful to avoid KPIs where you don't exclusively own the underlying thread (like bsd_timeout and the thread call KPI </kern/thread_call.h>). Changing the priority on system threads is not a good idea. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Threads within the kernel
by Terry Lambert 13 Dec '06

13 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Consider this an educational question for now. I'm considering switching things back to the way they were before, which aleviates the problem ... and thus the need for a solution, but I'm still interested in what's going on in the kernel. My observation is that there is a difference between threads associated with user processes and those "spawned" by a call to bsd_timeout(). There is this lengthy (~15 seconds) calculation that takes place in response to input from the user app. In a previous version of this program, that happened directly within the thread associated with the user process which called send() to give some input to the kernel extension. The effect was that send() blocked for those ~15 seconds but the rest of the system behaved normally. When this same calculation takes place within a thread "spawned" by bsd_timeout(), the whole system[1] seems to lock up. I can't even move the mouse. -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Dec 11, 2006, at 4:32 PM, Curtis Jones wrote: Anyway ... can anyone articulate the difference? You don't really want to do this; in addition to the priority inversion that Quinn talked about, you also have to worry that the operation is not happening in the context of the process that initiated it - it's happening in the kernel context instead. This is in fact why it had the rights to effectively take over the machine in the first place. This means that if your driver does this, it's going to do the operation with root privileges (which is likely not what you want), and it's going to do it in such a way that it won't be able to directly communicate with the user process (e.g. via copyin, copyout, etc.) because it won't have a handle on the users address space map to be able to do the job. In other words, bsd_timeout() should be used to wake up user processes whose own threads happen to be blocked on a wait channel (sleep/tsleep/ msleep address), and the work should be done in the thread that woke up once there was more work to do in behalf of the process requesting that it be done. If the user process needs more threads to do its thing, it should start them itself; this also prevents the process from running away with the store accidently, since only a privileged process is permitted to try. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Problem in Kernel Socket Read.
by Terry Lambert 13 Dec '06

13 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com The obvious cause for this is documented in the header: @result 0 on success, EWOULDBLOCK if non-blocking and operation would cause the thread to block, otherwise the errno error. Also, you should wean yourself off mixing up: ENOTSUP - Operation not supported EOPNOTSUPP - Operation not supported on socket -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Dec 11, 2006, at 3:07 AM, Quinn wrote: At 12:43 +0530 11/12/06, Nandan Nivgune wrote: I am trying to read and write using Kernel socket in Mac os X 10.4.7; using "Kpi_socket" Header file. It performs connect & write operation successfully while in case of reading data from socket, it returns with error: 35, Resource temporary unavailable. If you look in "/usr/include/sys/errno.h", you'll see that error 35 is EAGAIN which is equivalent to EWOULDBLOCK. This will not necessarily be true forever; the proper errno for sockets is EWOULDBLOCK, please use EWOULDBLOCK in case it gets changed out from under you at some point. the second is always for sockets (except for lchown; the POSIX standard is sort of broken for that one API). Generally EAGAIN and ENOTSUP are private to file calls. If you don't know what an fd you are calling is attached to (because you don't remember from when you got it), you can always call fstat to ask. You can confirm the non-blocking status of a socket using sock_isnonblocking. If the sock is, in fact, in non-blocking mode and you want it to be in blocking mode, use <x-man-page://2/ioctl> with the FIONBIO selector to set it to be blocking. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/ > Apple Developer Relations, Developer Technical Support, Core OS/ Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/tlambert%40apple.com This email sent to tlambert(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: Threads within the kernel
by Quinn 12 Dec '06

12 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Relations, Developer Technical Support, Core OS/Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 19:32 -0500 11/12/06, Curtis Jones wrote: Anyway ... can anyone articulate the difference? This is just a matter of thread priority. The kernel thread that's used to implement bsd_timeout has a high, fixed priority. And, on a single CPU Mac OS X system, if a thread of priority X is available to run, threads of a priority less than X do not run at all. If you're doing CPU intensive stuff, you have to make sure than you run it in a low priority thread. User space threads, by default, are what we call "time shared". That is, as the thread consumes more and more CPU time, the kernel depresses its priority to give other threads a chance. Kernel threads, by default, are of fixed priority. This email sent to site_archiver(a)lists.apple.com

1 0

Threads within the kernel
by Curtis Jones 12 Dec '06

12 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=KXZWR4Q5nDSENL3qJrwpObmVGH6lO5joV6X+ivjbYGLnFoQK/VM8w7uCYKk1OJ/L9PDczKHZCl2uIpwbod4ZDpF+bjo4pD4jHOw4Q/XYwc+GoSRl0xDrSflxgdLEk+0Gm625Vn8c3NQvSJW0erYkjHOaOQGyvSE1HPlmIvtJf9M= Consider this an educational question for now. I'm considering switching things back to the way they were before, which aleviates the problem ... and thus the need for a solution, but I'm still interested in what's going on in the kernel. My observation is that there is a difference between threads associated with user processes and those "spawned" by a call to bsd_timeout(). There is this lengthy (~15 seconds) calculation that takes place in response to input from the user app. In a previous version of this program, that happened directly within the thread associated with the user process which called send() to give some input to the kernel extension. The effect was that send() blocked for those ~15 seconds but the rest of the system behaved normally. When this same calculation takes place within a thread "spawned" by bsd_timeout(), the whole system[1] seems to lock up. I can't even move the mouse. Anyway ... can anyone articulate the difference? -Curtis [1] It appears that once or twice the WindowServer gets to update the window a bit. The button that starts this process changes color to reflect the release of the mouse after a few seconds. Very slow. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Problem in Kernel Socket Read.
by Quinn 11 Dec '06

11 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com The obvious cause for this is documented in the header: @result 0 on success, EWOULDBLOCK if non-blocking and operation would cause the thread to block, otherwise the errno error. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Relations, Developer Technical Support, Core OS/Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 12:43 +0530 11/12/06, Nandan Nivgune wrote: I am trying to read and write using Kernel socket in Mac os X 10.4.7; using "Kpi_socket" Header file. It performs connect & write operation successfully while in case of reading data from socket, it returns with error: 35, Resource temporary unavailable. If you look in "/usr/include/sys/errno.h", you'll see that error 35 is EAGAIN which is equivalent to EWOULDBLOCK. You can confirm the non-blocking status of a socket using sock_isnonblocking. If the sock is, in fact, in non-blocking mode and you want it to be in blocking mode, use <x-man-page://2/ioctl> with the FIONBIO selector to set it to be blocking. This email sent to site_archiver(a)lists.apple.com

1 0

Problem in Kernel Socket Read.
by Nandan Nivgune 11 Dec '06

11 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thread-index: Accc89SgciSpjhOoRR6m/jd/AMwSEQ== Thread-topic: Problem in Kernel Socket Read. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: waiting queue
by Jim Magee 08 Dec '06

08 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Dec 6, 2006, at 1:37 AM, Quinn wrote: --Jim _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 11:11 -0800 5/12/06, Michael Smith wrote: As a general rule, if you believe that your hardware is reliable, use THREAD_UNINT to be safe. There's one gotcha here. Because of the way user clients are implementation, you shouldn't use THREAD_UNINT while blocking in a user client if that user client has wired down any memory from the client process. If you do so, there will be no way to kill the task. I've included the gory details below. This was on 10.3.x, so YMMV on later systems. Yes. You may only user THREAD_UNINT is you are "guaranteed" that the block is of a bounded duration. I put the guaranteed in quotes, because nothing is certain in this life. But if there is any (reasonable) likelihood that it might not get woken up (known flakey hardware, or user-removable device with no wakeup, etc...) then you either need to make the wait THREAD_INTERRUPTIBLE or set a timer for the wait. Oops, I guess I forgot to mention that in the previous note. All waits, even THREAD_UNINT waits, will return/call their continuation on wait timeout. This email sent to site_archiver(a)lists.apple.com

1 0

Re: waiting queue
by Jim Magee 08 Dec '06

08 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Absolutely correct. --Jim _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Just a clarification on the subtleties of the THREAD_XXX interruptible flags. On Dec 5, 2006, at 11:11 AM, Michael Smith wrote: I have one question left where the documentation does not tell me the whole story: #define THREAD_UNINT 0 /* not interruptible The sleep is guaranteed to return, and only when someone explicitly wakes it up. It will return THREAD_AWAKENED. #define THREAD_INTERRUPTIBLE 1 /* may not be restartable */ The sleep is guaranteed to return, but may do so for various reasons (e.g. a signal has been sent to the thread). It will return THREAD_AWAKENED if it was woken up explicitly, and THREAD_INTERRUPTED if it was woken for some other reason. If it returns THREAD_RESTART (or anything else) you should restart the sleep. Actually, the distinction is more subtle than that. You use this state when you want the wait to return only when someone explicitly wakes you up or the current process/thread is being killed. The only signal that interrupts this state is a "kill -9". Providing this value means that your operation may not be cleanly restartable for the current process/thread (you might lose the result of the operation as detectable only from the current client connection), but since it is exiting anyway, you're willing to cut the operation short in that case. Of course, the "system as a whole" has to be able to cleanly deal with/recover from this interruption (the device as a whole must still be a consistent/recoverable state after the interruption). This state is sticky. Once a thread is marked for termination, each and every interruptible wait will return immediately until the thread actually exits. #define THREAD_ABORTSAFE 2 /* abortable safely */ The sleep is not guaranteed to return. This is a promise on your part that the thread can be aborted (destroyed) while asleep without affecting your code. Generally in the driver environment this is OK to use while you hold no locks or resources (e.g. whilst waiting to get into your driver), but otherwise you should avoid it. You are not notified when the thread is aborted; it just never comes back. All waits return (unless you directed them to a continuation routine - but in that case the continuation routine will always be called for each wait). Threads never just disappear out from under you. What this value means is that you are willing to interrupt the current wait (and proceed to unwind back to the system call/trap) in the face of "any" user signal. Many of those syscalls/traps may try to restart the operation after the signal is handled. Your driver must be able to deal with that if you use this value during your wait. But you must provide this value for any unbounded wait - lest you pend such signals forever. The Mach operation that is used to poke a thread in the case of such non-fatal signals is a thread_abort_safely() call (only abort the current wait if the thread is waiting "safely"). Hence the origin of the flag's name. This state is not guaranteed to be sticky. Once one wait detects it, the code in its return path to the user must be able to unwind and avoid subsequent unbounded wait calls. So, in order to unwind cleanly, you likely have to return an error code to you caller that will allow them to understand you are interrupting the current call, and they do it to their caller, etc..., etc..., etc... This email sent to site_archiver(a)lists.apple.com

1 0

Re: Ports lost after reboot..
by Quinn 06 Dec '06

06 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com There are two common techniques to solve this problem. <http://developer.apple.com/documentation/DeviceDrivers/Conceptual/IOKitFund…> <http://developer.apple.com/technotes/tn2005/tn2083.html> S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Relations, Developer Technical Support, Core OS/Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 9:35 +0530 6/12/06, sanjay yaragatti wrote: There is no hardware.Its a driver for virtual com port. Hence cant load it automatically when the hardware is detected.. 1. Package your driver entirely within I/O Kit and have I/O Kit load your driver by specifying a match on IOResources. If you search for "virtual" on the following web page, there's a detailed discussion of this. 2. Explicitly load your driver at boot time using a launchd daemon (or a startup item if you need to support pre-10.4). The following technote introduces these concepts and has links to more documentation. This email sent to site_archiver(a)lists.apple.com

1 0