July 2005 - Darwin-kernel - lists.apple.com

Re: msleep() in NKE?
by Frank Thomas 29 Jul '05

29 Jul '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=PiF72bTVCTvauB2hDPfHu+Q7/HVSuQcqVgHu7kudJTcHsmdCb8kQ6s4jOLEvUeo+U2EtHeRN/yuAD+urBaYGCNfNiu8I6Qzc1AjmUyzllRzcBwD71s2TmLNldG1WFjySISClPFe8qVpL1vjQiWjv+/Z742/OEHmYF05tIoUye08= On 7/28/05, Allan Hoeltje <ahoeltje(a)lmi.net> wrote: > Being a newbie at kernel extensions myself, I am reluctant to suggest that I > may have an answer but, what they hey, I'll take a chance at it. > > I also am doing a KEXT that inspects IP packets and converses with a daemon > (a launch agent) via sockets. I do not have everything in place yet but > mostly it is working. > > If you need to have a user app or daemon process a packet before it is sent > up/down the IP stack then you will need to remove the packet from the IP > stack in your kext input/output function (swallow the packet with a "return > EJUSTRETURN") and send it to your user app via ctl_enqueuedata. Have your > user app do it's thing at user speed and then send it back to your kext. > The kext repackages it into an mbuf and injects it (ipf_inject_input or > ipf_inject_output) back into the IP stack. > > This assumes that the packet is not a vital part of some important stream of > packets and can be processed on its own. If it is part of a stream and > order is important to the remote host then you will need to swallow all of > them and inject them later to keep them in order. > > It is probably not a good idea to put the kernel to sleep. :-) > > Hope this is mostly correct and helps! > > -Allan Allan, Thanks for your response, we half expected to get this as our first response. We actually had tried this...it was done before the sleep. But we found it added a significant latency to the connection. We were doing almost exactly what you indicated. Take an HTTP stream for example. Say we removed the "GET" packet from the stream. This would hold the browser. We would do our user space stuff and send it back down to demon on the sys_prot socket. When the request was recv'd at the kernel we would re_inject the approiate packet. However, it would add about a second delay to the request. For each request including all images on the page etc this would really add up to a very noticable delay. We felt like we were back on dialup. Have you seen anythhing like that? While the sleep might not be the best way to handle the delay it gave us a noticable boost is speed for the most part, only on occasion where we have to prompt userland do we have a problem becuase the delay could be pretty long. - Frank _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: POSIX Async I/O
by Terry Lambert 28 Jul '05

28 Jul '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com async io is there in Tiger. look at: bsd/sys/aio.h bsd/kern/kern_aio.c Mea culpa. It is the case that aio.h already documents that the O_DSYNC argument is not supported. Sorry about that. I don't have a copy of POSIX handy to confirm this, but the O'Reilly book "POSIX.4: Programming for the Real World" says (page 343): "fdatasync() may be implemented as a call to fsync(), in which case fdatasync() will be no faster than fsync()." This implies that it might be legal under POSIX for Apple to do as the quoted text says above for the aio_fsync() case. If that is the main difference between Apple's implementation and POSIX compliance for this symbol (_POSIX_ASYNCHRONOUS_IO), and if the quoted sentence above were true, then it would be nice if Apple could put the needed shims into headers/source so that the implementation could become POSIX compliant for that symbol. Since my source code does check for the standard POSIX symbol, as part of trying to be clean portable code, that tweak would eliminate the need for me to special case the Apple platform (which is all "nice to have", rather than absolutely essential). It's just one of those strings you don't really want to pull. 8-(. -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Jul 28, 2005, at 3:38 PM, Ran Atkinson wrote: On 28 Jul 2005, at 16:54, Terry Lambert wrote: On Jul 28, 2005, at 9:28 AM, Jerry Cottingham wrote: On Jul 28, 2005, at 8:57 AM, Ran Atkinson wrote: I have some existing UNIX-based (i.e. not Carbon/Cocoa) software that uses the POSIX Asynchronous I/O interfaces. These seem not to be present in Darwin (MacOS X 10.4), at least if unistd.h is to be believed. POSIX requires that _POSIX_ASYNCHRONOUS_IO be defined to 200112L in <unistd.h> *ONLY* if you fully support all aspects of the API features for that option group. The MacOS X implementation supports all important aspects of the [AIO] feature group, but not all possible aspects. In particular, we don't support the O_DSYNC argument to aio_fsync(). I considered doing that, when I went to set the <unistd.h> value, but there are a couple of implementational problems with a "just do it" approach: (1) It's actually possible to write code that's sensitive to whether or not it's implemented that way (trust me: I went out of my way to write some). (2) There's specific reference to _POSIX_SYNCHRONIZED_IO when you do this, so that would have to be there, too. (3) If we defined a value for O_DSYNC now, we'd have to pick its value very, very carefully - it's used by other APIs, e.g. open(2), which would have to behave correctly, and which would need to have it not collide with other flags definitions. For right now, I think we'll have to stick with "Use the _POSIX_ASYNCHRONOUS_IO test in general, and test for MacOS X, too, if you aren't using O_DSYNC". This approach will keep working if or when we support _POSIX_ASYNCHRONOUS_IO fully. This email sent to site_archiver(a)lists.apple.com

1 0

Re: msleep() in NKE?
by Allan Hoeltje 28 Jul '05

28 Jul '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com User-agent: Microsoft-Entourage/10.1.4.030702.0 Being a newbie at kernel extensions myself, I am reluctant to suggest that I may have an answer but, what they hey, I'll take a chance at it. I also am doing a KEXT that inspects IP packets and converses with a daemon (a launch agent) via sockets. I do not have everything in place yet but mostly it is working. If you need to have a user app or daemon process a packet before it is sent up/down the IP stack then you will need to remove the packet from the IP stack in your kext input/output function (swallow the packet with a "return EJUSTRETURN") and send it to your user app via ctl_enqueuedata. Have your user app do it's thing at user speed and then send it back to your kext. The kext repackages it into an mbuf and injects it (ipf_inject_input or ipf_inject_output) back into the IP stack. This assumes that the packet is not a vital part of some important stream of packets and can be processed on its own. If it is part of a stream and order is important to the remote host then you will need to swallow all of them and inject them later to keep them in order. It is probably not a good idea to put the kernel to sleep. :-) Hope this is mostly correct and helps! -Allan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: POSIX Async I/O
by Ran Atkinson 28 Jul '05

28 Jul '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Jul 28, 2005, at 8:57 AM, Ran Atkinson wrote: I have some existing UNIX-based (i.e. not Carbon/Cocoa) software that uses the POSIX Asynchronous I/O interfaces. These seem not to be present in Darwin (MacOS X 10.4), at least if unistd.h is to be believed. async io is there in Tiger. look at: bsd/sys/aio.h bsd/kern/kern_aio.c In particular, we don't support the O_DSYNC argument to aio_fsync(). Mea culpa. It is the case that aio.h already documents that the O_DSYNC argument is not supported. Sorry about that. I don't have a copy of POSIX handy to confirm this, but the O'Reilly book "POSIX.4: Programming for the Real World" says (page 343): "fdatasync() may be implemented as a call to fsync(), in which case fdatasync() will be no faster than fsync()." This implies that it might be legal under POSIX for Apple to do as the quoted text says above for the aio_fsync() case. If that is the main difference between Apple's implementation and POSIX compliance for this symbol (_POSIX_ASYNCHRONOUS_IO), and if the quoted sentence above were true, then it would be nice if Apple could put the needed shims into headers/source so that the implementation could become POSIX compliant for that symbol. Since my source code does check for the standard POSIX symbol, as part of trying to be clean portable code, that tweak would eliminate the need for me to special case the Apple platform (which is all "nice to have", rather than absolutely essential). Thanks, Ran _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On 28 Jul 2005, at 16:54, Terry Lambert wrote: On Jul 28, 2005, at 9:28 AM, Jerry Cottingham wrote: POSIX requires that _POSIX_ASYNCHRONOUS_IO be defined to 200112L in <unistd.h> *ONLY* if you fully support all aspects of the API features for that option group. The MacOS X implementation supports all important aspects of the [AIO] feature group, but not all possible aspects. -- Terry This email sent to site_archiver(a)lists.apple.com

1 0

Re: selrecord/selwakeup now unusable ?
by Terry Lambert 28 Jul '05

28 Jul '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi. I looked at the fsevents code, but cannot figure out how to use it. I can't find any documentation or coding example. Help! Hi Christer; -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Jul 28, 2005, at 2:26 PM, Christer Bernérus wrote: I'm finally trying to port the Coda file system kernel extension to Tiger. Can anyone enlighten me how I'm supposed to do instead of using selrecord/selwakeup ? It doesn't seem to be supported anymore. At least the necessary selinfo structure is not defined in the kernel framework, and if I try the ugly trick of copying the definition from the xnu source, that just leads to undefined wait queue structures. This suggests to me that I should follow another path. What problem are you trying to solve that makes you think you need to directly access selrecord/selwakeup? This is basically the select mechanism internals. If you can define the problem, someone can probably give you a way to solve it that fits into the context of the KPI. For example, if you are attempting a read-with-timeout, you should probably llok at the changes that were made to the published SMB implementation when smb/kernel/netsmb/smb_trantcp.c lost access to the select internals. That was only intended as an example - if it's not applicable, and you can tell us what you are doing, we could probably come up with a better example of code for you. This email sent to site_archiver(a)lists.apple.com

1 0

Re: POSIX Async I/O
by Ran Atkinson 28 Jul '05

28 Jul '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On 28 Jul 2005, at 16:54, Terry Lambert wrote: On Jul 28, 2005, at 9:28 AM, Jerry Cottingham wrote: On Jul 28, 2005, at 8:57 AM, Ran Atkinson wrote: I have some existing UNIX-based (i.e. not Carbon/Cocoa) software that uses the POSIX Asynchronous I/O interfaces. These seem not to be present in Darwin (MacOS X 10.4), at least if unistd.h is to be believed. async io is there in Tiger. look at: bsd/sys/aio.h bsd/kern/kern_aio.c In particular, we don't support the O_DSYNC argument to aio_fsync(). -- Terry Thank you. That is very helpful. In the long term, it would be helpful to application developers if the header file (sys/aio.h) had a comment or two indicating which parts of the POSIX spec are not yet complied with. That way an application programmer would have a good chance of figuring out whether an application needed rework to compile under MacOS X or whether it would be OK as is (e.g. because O_DSYNC) were not used by the application). Just for information, I did file this as a bug, though maybe it is a "documentation issue" rather than an actual software bug given what I just learned from you. The bug ID is 4196956. Thanks again, Ran rja(a)inet.org _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… POSIX requires that _POSIX_ASYNCHRONOUS_IO be defined to 200112L in <unistd.h> *ONLY* if you fully support all aspects of the API features for that option group. The MacOS X implementation supports all important aspects of the [AIO] feature group, but not all possible aspects. This email sent to site_archiver(a)lists.apple.com

1 0

msleep() in NKE?
by Frank Thomas 28 Jul '05

28 Jul '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=GwHOSrEeoAYtBxIjTNOnR1+nPZ8ubifNxmfuYc67yZ29tsNsdI5MQb9llHFcZo3Ra30kBYbMQMUC+IhHP/6122HW2/jxHMXyxbvMezjn63zE7K2JDQY3BAh4htLqNLTQ1T8LyWFRZaQoBLDbRo0HjNbo4WUWGnxwQd+z5JiWHew= I'm working on an ipfilter NKE that needs to defer processing on certain TCP connections. Basically the NKE sends selected data up to a userland daemon that inspects the data on the connection and optionally invokes a user interface to prompt the user for how to handle the data. The packet is then released outbound and everything continues on as normal. I'm currently using msleep() in the ipfilter outbound callback function to delay, and waking up the channel when the daemon responds. This is all working great so far, but there's a problem when the daemon/UI process takes a long time to return (e.g. if the user takes a while to handle the prompts). It looks like when the sleep runs for a certain amount of time, the network stack deadlocks until the msleep() call times out. I've confirmed this by varying the sleep anywhere from 10 milliseconds to 30 seconds. Even when all I do is sleep in the output function (with literally no other processing) this can be reproduced. This happens whether I call msleep() once with a long timeout, or repeatedly with short timeouts. The symptoms are that when the sleep hits this threshold (which seems to vary depending on the volume of traffic on the system) no traffic reaches the network stack (including loopback traffic) and no traffic is sent over the wire until the sleep times out or is woken up. However, I've noticed that sleeping in and of itself doesn't appear create any problems - tests seem to have proven that traffic on another connection can still be passed even while the hook function is sleeping. Anyone have any ideas on how to keep the rest of the network stack from deadlocking like this? The Apple docs suggest that msleep() is safe within an NKE and do not clearly present any alternatives. Due to the purpose of the application, dropping the packet and reinjecting it (via returning EJUSTRETURN and using ipf_inject_output) is not a viable option at this point, so I'd really like to avoid rearchitecting the application (especially since the bulk of the application works flawlessly). - Frank _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

selrecord/selwakeup now unusable ?
by Christer Bernérus 28 Jul '05

28 Jul '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

cpu_number() in Tiger...
by Herb Petschauer 28 Jul '05

28 Jul '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=H34wl2aLxQbUncRqals7e7vSWvcEonsIUc9IC94dsQeWVAB2fg7R81d0zQRHXG+X/45OmBqf1RDrZhwNVfDqNq0hfQ+nr1aW30PSU7TQSZbSSRkEG61nq94OIDUS+Iv3RxAu1XmayGJdiUD2WcJRG+Nq6OU82RNjnG/Tnucnmo0= Looks like today is my Tiger port question day. Is there a way of getting cpu_number(), i.e. what cpu is the current thread running on? I can see it in struct thread's machine.PerProc.cpu_number but so far I haven't been able to find an accessor for it. Thanks! Cheers, H. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: POSIX Async I/O
by Terry Lambert 28 Jul '05

28 Jul '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com I have some existing UNIX-based (i.e. not Carbon/Cocoa) software that uses the POSIX Asynchronous I/O interfaces. These seem not to be present in Darwin (MacOS X 10.4), at least if unistd.h is to be believed. async io is there in Tiger. look at: bsd/sys/aio.h bsd/kern/kern_aio.c In particular, we don't support the O_DSYNC argument to aio_fsync(). -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Jul 28, 2005, at 9:28 AM, Jerry Cottingham wrote: On Jul 28, 2005, at 8:57 AM, Ran Atkinson wrote: POSIX requires that _POSIX_ASYNCHRONOUS_IO be defined to 200112L in <unistd.h> *ONLY* if you fully support all aspects of the API features for that option group. The MacOS X implementation supports all important aspects of the [AIO] feature group, but not all possible aspects. This email sent to site_archiver(a)lists.apple.com

1 0