September 2007 - Darwin-dev

Re: Volume header needs minor repair
by Vlad Beffa 08 Sep '07

08 Sep '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: EVFILT_SIGNAL - observe "standard" signals before they are delivered?
by Terry Lambert 07 Sep '07

07 Sep '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Sep 7, 2007, at 12:43 AM, Jonas Maebe wrote: On 07 Sep 2007, at 01:19, Terry Lambert wrote: A signal starts as a Mach exception, which can be intercepted and averted, if you hold the exception port for the process and know what you are doing. For example, gdb only partially knows what it's doing; if you don't turn off "start-with-subshell", which is on by default, for example, gdb wil hold the exception port on the subshell rather than the process being debugged, and you will not be able to override signals, since you will only see them in gdb via ptrace, rather than as a Mach exception. Does this mean that this could be used somehow as a workaround to debug unix signals in gdb? Yes. It is how you debug programs with issues around their signal handling, or signals in general. Or is there any other workaround besides "sleep inside your signal handler and attach with gdb only after the signal has been triggered"? Not that I know of; on a standard BSD system, ptrace could let you do this, but Mach semantics are expected by gdb on Mac OS X. This email sent to site_archiver(a)lists.apple.com

1 0

Re: EVFILT_SIGNAL - observe "standard" signals before they are delivered?
by Jonas Maebe 07 Sep '07

07 Sep '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 07 Sep 2007, at 01:19, Terry Lambert wrote: Jonas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… A signal starts as a Mach exception, which can be intercepted and averted, if you hold the exception port for the process and know what you are doing. For example, gdb only partially knows what it's doing; if you don't turn off "start-with-subshell", which is on by default, for example, gdb wil hold the exception port on the subshell rather than the process being debugged, and you will not be able to override signals, since you will only see them in gdb via ptrace, rather than as a Mach exception. Does this mean that this could be used somehow as a workaround to debug unix signals in gdb? Or is there any other workaround besides "sleep inside your signal handler and attach with gdb only after the signal has been triggered"? This email sent to site_archiver(a)lists.apple.com

1 0

Re: EVFILT_SIGNAL - observe "standard" signals before they are delivered?
by Terry Lambert 06 Sep '07

06 Sep '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com The man page says: Short answer: - Long answer: - - -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Sep 5, 2007, at 12:00 PM, Jim Correia wrote: EVFILT_SIGNAL Takes the signal number to monitor as the identifier and returns when the given signal is delivered to the process. This coexists with the signal() and sigaction() facili- ties, and has a lower precedence. The filter will record all attempts to deliver a signal to a process, even if the signal has been marked as SIG_IGN. Event notification happens after normal signal delivery processing. data returns the number of times the signal has occurred since the last call to kevent(). This filter automatically sets the EV_CLEAR flag internally. Am I interpreting that correctly to mean that a signal whose default behavior is process termination (SIGSEGV, for example) will not be observable by the kevent mechanism? No, it just means that you don't get to see the signal before the process does and prevent its delivery; you are only an observer. It also means that there is no guarantee of ordering, so you're not allowed to make any assumptions about it. Signals are very complicated on MacOS X due to interaction with the Mach AST model (True64 UNIX has the same issues, as it is also Mach- based, and uses the AST mechanism to deliver signals). A signal starts as a Mach exception, which can be intercepted and averted, if you hold the exception port for the process and know what you are doing. For example, gdb only partially knows what it's doing; if you don't turn off "start-with-subshell", which is on by default, for example, gdb wil hold the exception port on the subshell rather than the process being debugged, and you will not be able to override signals, since you will only see them in gdb via ptrace, rather than as a Mach exception. After it goes to the task exception port because it hasn't been intercepted at the Mach exception level, it becomes an AST_BSD, and then is only handled when a thread stops being uninterruptible. It becomes interruptible in Mach, which throws the AST at that point. This means there can be considerable latency, if an operation is uninterruptible, and it can also mean that the operation that was in progress at the time of the exception being thrown may in fact have completed successfully, rather than being interrupted (so, for example, you seldom see a disk read getting interrupted by a signal and returning EINTR). After this, if the operation was interruptible (e.g. a tsleep/msleep operation on a tty wait queue or whatever), the operation is interrupted, and goes to return an EINTR up to the user/kernel boundary (an error on the BSD system call). When we run up to the user/kernel boundary, we return to an address other than the address requested, with a bunch of state on the stack (this boils down to a "siginfo" structure with potentially multiple ucontext, mcontext, and other state structures dealing with process state prior to the running of a signal handler). The address we return to is several bytes off the actual system call stub return address, and it jumps to the signal handler for the signal in question (based, in part, on some contents of the siginfo structure which must be treated as opaque by user space application). The handler runs in user space, then calls a system call that says "put my state back to what it was before the handler fired", and is passed the context information we passed to user space in the first place ("sigreturn"). At this point, the thread state is restored, and we return the EINTR to the system call caller. The thrown AST is caught in kern_sig.c, which is what throws the KNOTE that causes the kevent to get delivered to you. Unless it gets here, you don't get the event. So you won't see the signal if you intentionally intercept it (mostly by Mach means) prior to the thrown AST being turned into a "real" signal, which in turn will get delivered when it's delivered, which can be before or after your event gets delivered, depending on the code path on which the exception happened. You can ALSO not see a signal if the signal would have been EINTR, and the operation was not an explicit signal send, AND the underlying operation completed successfully because all the work it had to do happened before the process (Mach task, actually) became interruptible. Our UNIX certification has the same permanent interpretation on this that True64 UNIX has, and that a couple of vendors have had on their signal implementations in the past. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Poor performance of pthreads across cpu dies.
by Andrew Gallatin 05 Sep '07

05 Sep '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Terry Lambert writes: > On Sep 5, 2007, at 1:21 PM, Andrew Gallatin wrote: > > Terry Lambert writes: > >> From my reading of things, the issue as to whether or not you can > >> target multiple CPUs with MSI has more to do with contiguous vs. non- > >> contiguous vector assignment (which means that it's not possible in > >> older Linux kernels). The MSI-X implementation overcomes this in > >> older Linux kernels due to independent configuration of message data > >> and address. BUT, if you are willing to perform contiguous > >> allocations, it's not a problem for MSI to do the same. > > > > I'm not sure what linux kernels have to do with it. > > The limitation is an old Linux one, according to Intel. > > If you were to allocate 4 contiguous vectors, each vector could target > a different ID, and the set of vectors could be associated with a > single device. > > > The problem with > > multiple messages in MSI is that all messages have the same message > > address, but just different message data. Using the same message > > address means that all interrupts will be delivered to the same > > APIC, and hence to the same CPU. However, maybe this isn't a problem > > on darwin/OSX, since all hardware interrupts are delivered to CPU 0 > > anyway... > > I think you missed where Mike pointed out that delivery is not as > straight-forward as I initially said (I was mistaken, and Mike was > correct here). That is what I meant by "maybe this isn't a problem on darwin/OSX" :) However, it is a problem in that our NIC uses MSI-X, and not multiple MSI messages. Drew _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Poor performance of pthreads across cpu dies.
by Terry Lambert 05 Sep '07

05 Sep '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com >From my reading of things, the issue as to whether or not you can target multiple CPUs with MSI has more to do with contiguous vs. non- contiguous vector assignment (which means that it's not possible in older Linux kernels). The MSI-X implementation overcomes this in older Linux kernels due to independent configuration of message data and address. BUT, if you are willing to perform contiguous allocations, it's not a problem for MSI to do the same. The limitation is an old Linux one, according to Intel. The problem with multiple messages in MSI is that all messages have the same message address, but just different message data. Using the same message address means that all interrupts will be delivered to the same APIC, and hence to the same CPU. However, maybe this isn't a problem on darwin/OSX, since all hardware interrupts are delivered to CPU 0 anyway... -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Sep 5, 2007, at 1:21 PM, Andrew Gallatin wrote: Terry Lambert writes: I'm not sure what linux kernels have to do with it. If you were to allocate 4 contiguous vectors, each vector could target a different ID, and the set of vectors could be associated with a single device. I think you missed where Mike pointed out that delivery is not as straight-forward as I initially said (I was mistaken, and Mike was correct here). This email sent to site_archiver(a)lists.apple.com

1 0

Re: Poor performance of pthreads across cpu dies.
by Andrew Gallatin 05 Sep '07

05 Sep '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Terry Lambert writes: > On Sep 5, 2007, at 6:26 AM, Andrew Gallatin wrote: > > Terry Lambert writes: > >> On Sep 4, 2007, at 6:09 AM, Andrew Gallatin wrote: > >>> The *NIC* has looked at it, and can easily hash different > >>> connections > >>> to several MSI-X interrupt handlers, which are then each bound to > >>> different CPUs (or groups of CPUs). > >> > >> I am told we support this (Message Signalled Interrupts) for the > >> latest Intel OS releases, and that it is in fact the preferred method > >> of interrupt delivery these days, for hardware capable of supporting > >> it. I wasn't able to find the tech note, which is supposed to have > >> been published already, according to the author, so you might want to > >> contact DTS. > > > > MSI is different from MSI-X. I figured out how to use MSI from the > > headers / source last winter, but AFAIK, darwin/macosx does not yet > > support MSI-X, which is required to have multiple interrupt vectors > > per device which are bound to different CPUs. > > > Like I said, it's probably time to stop speculating, and ask DTS for > the Tech Note. I looked at the (tiger) header files.. But a tech note would be appreciated. > - > > According to the public documentation available to me, the advantages > of MSI-X over MSI are: > > - Support a larger maximum number of vectors per function > > - Provide the ability for system software to configure each vector > with an independent message address and message data, > specified by a table that resides in Memory Space > > - Per-vector masking is an optional extension of MSI but a required > feature for MSI-X. > > From my reading of things, the issue as to whether or not you can > target multiple CPUs with MSI has more to do with contiguous vs. non- > contiguous vector assignment (which means that it's not possible in > older Linux kernels). The MSI-X implementation overcomes this in > older Linux kernels due to independent configuration of message data > and address. BUT, if you are willing to perform contiguous > allocations, it's not a problem for MSI to do the same. I'm not sure what linux kernels have to do with it. The problem with multiple messages in MSI is that all messages have the same message address, but just different message data. Using the same message address means that all interrupts will be delivered to the same APIC, and hence to the same CPU. However, maybe this isn't a problem on darwin/OSX, since all hardware interrupts are delivered to CPU 0 anyway... Drew _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Poor performance of pthreads across cpu dies.
by Terry Lambert 05 Sep '07

05 Sep '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com I am told we support this (Message Signalled Interrupts) for the latest Intel OS releases, and that it is in fact the preferred method of interrupt delivery these days, for hardware capable of supporting it. I wasn't able to find the tech note, which is supposed to have been published already, according to the author, so you might want to contact DTS. MSI is different from MSI-X. I figured out how to use MSI from the headers / source last winter, but AFAIK, darwin/macosx does not yet support MSI-X, which is required to have multiple interrupt vectors per device which are bound to different CPUs. - - Support a larger maximum number of vectors per function - Provide the ability for system software to configure each vector with an independent message address and message data, specified by a table that resides in Memory Space - Per-vector masking is an optional extension of MSI but a required feature for MSI-X. -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Sep 5, 2007, at 6:26 AM, Andrew Gallatin wrote: Terry Lambert writes: On Sep 4, 2007, at 6:09 AM, Andrew Gallatin wrote: The *NIC* has looked at it, and can easily hash different connections to several MSI-X interrupt handlers, which are then each bound to different CPUs (or groups of CPUs). Like I said, it's probably time to stop speculating, and ask DTS for the Tech Note. According to the public documentation available to me, the advantages of MSI-X over MSI are: >From my reading of things, the issue as to whether or not you can target multiple CPUs with MSI has more to do with contiguous vs. non- contiguous vector assignment (which means that it's not possible in older Linux kernels). The MSI-X implementation overcomes this in older Linux kernels due to independent configuration of message data and address. BUT, if you are willing to perform contiguous allocations, it's not a problem for MSI to do the same. NB: This is according to Tom L. Nguyen of Intel's "The MSI Driver Guide HOWTO" for Linux, revised 12 Feb 2004 by Martine Silbermann of HP. This email sent to site_archiver(a)lists.apple.com

1 0

EVFILT_SIGNAL - observe "standard" signals before they are delivered?
by Jim Correia 05 Sep '07

05 Sep '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com The man page says: Thanks, Jim _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… EVFILT_SIGNAL Takes the signal number to monitor as the identifier and returns when the given signal is delivered to the process. This coexists with the signal() and sigaction() facili- ties, and has a lower precedence. The filter will record all attempts to deliver a signal to a process, even if the signal has been marked as SIG_IGN. Event notification happens after normal signal delivery processing. data returns the number of times the signal has occurred since the last call to kevent(). This filter automatically sets the EV_CLEAR flag internally. Am I interpreting that correctly to mean that a signal whose default behavior is process termination (SIGSEGV, for example) will not be observable by the kevent mechanism? This email sent to site_archiver(a)lists.apple.com

1 0

Re: Poor performance of pthreads across cpu dies.
by Shawn Erickson 05 Sep '07

05 Sep '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Sep 5, 2007, at 6:26 AM, Andrew Gallatin wrote: No they can. -Shawn _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… And I think that a single driver is not allowed to have multiple workloops. This email sent to site_archiver(a)lists.apple.com

1 0