Re: Poor performance of pthreads across cpu dies.
Re: Poor performance of pthreads across cpu dies.
- Subject: Re: Poor performance of pthreads across cpu dies.
- From: Andrew Gallatin <email@hidden>
- Date: Wed, 5 Sep 2007 09:26:32 -0400 (EDT)
Terry Lambert writes:
> On Sep 4, 2007, at 6:09 AM, Andrew Gallatin wrote:
> > The *NIC* has looked at it, and can easily hash different connections
> > to several MSI-X interrupt handlers, which are then each bound to
> > different CPUs (or groups of CPUs).
>
> I am told we support this (Message Signalled Interrupts) for the
> latest Intel OS releases, and that it is in fact the preferred method
> of interrupt delivery these days, for hardware capable of supporting
> it. I wasn't able to find the tech note, which is supposed to have
> been published already, according to the author, so you might want to
> contact DTS.
MSI is different from MSI-X. I figured out how to use MSI from the
headers / source last winter, but AFAIK, darwin/macosx does not yet
support MSI-X, which is required to have multiple interrupt vectors
per device which are bound to different CPUs.
In fact, I think MSI-X and support for multiple interrupt handlers per
device is in direct conflict with the IOKit workloop design concept.
If all the interrupt vectors activated the same workloop, MSI-X would
be pointless. Having all the vectors serialized into a single
workloop would loose any hope parallelism. And I think that a single
driver is not allowed to have multiple workloops.
> > Microsoft has though. It is called "receive side scaling" or RSS.
> > The cleverness is that the NIC and the host has the connections to the
> > same (sets of) CPUs.
> >
> > Even in the absence of RSS, multiple MSI-X interrupt handlers, etc,
> > you can get a decent approximation of a crystal ball by having the
> > ability to statically bind a single interrupt and a set of hot threads
> > to set of CPUs. Eg, the administrator is the crystal ball.
>
> It's a little more complex than that when you throw power management
> into the Mix, but it's more or less up to the driver author to do the
> RSS implementation on top of MSI.
Not MSI, MSI-X, which is AFAIK, missing in darwin/macosx.
> Of course, this requires relatively smart hardware to take full
I think all 10GbE NICs on the market today can do this, as well
as some 1GbE NICs.
> advantage of it (e.g. Microsoft want segment offloading in the cards,
> plus scatter/gather, etc.).
These features are supported in nearly every OS besides MacOSX. Heck,
you can find TSO support on almost every 1Gbe NIC (including the Intel
NICs built into Intel Apple hardware). I opened an enhancement
request for TSO over 2 years ago (4131204).
Drew
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden