Re: TCP, and clamping MSS..?
Re: TCP, and clamping MSS..?
- Subject: Re: TCP, and clamping MSS..?
- From: Andrew Gallatin <email@hidden>
- Date: Fri, 12 Jan 2007 11:55:37 -0500 (EST)
John Heffner writes:
> Andrew Gallatin wrote:
> > Vincent Lubet writes:
> > > Andrew,
> > >
> > > This is an optimization that is simple missin in xnu -- thanks for
> > > bringing it to our attention.
> > >
> > > Feel free to file an enhancement request at http://
> > > bugreporter.apple.com/ to make sure this won't fall through the cracks.
> > >
> >
> > Thanks Vince.. I've filed problem id 4919145.
> >
> > FWIW, it looks like the MSS clamping code was removed sometime between
> > 10.2 and 10.3. I've never really understood why it helps so much.
> > This isn't just my NIC that it helps for. See the thread starting at
> > http://lists.freebsd.org/pipermail/freebsd-net/2004-January/002594.html
> > where I actually proposed removing it, and found that it really did
> > help!
> >
> > Do you have any intuition on why keeping things nicely aligned helps
> > so much? Linux, on the same hardware, does not need to clamp the MTU,
> > and easily maxes out the 10GbE link with far lower CPU utilization.
> > Is the BSD stack just that much more sensitive to alignment?
>
> FWIW, Linux does exhibits this effect, too, though you probably won't
> see it if it's doing TSO. I observed exactly this phenomenon on a
> Pentium 3 back in the days when GigE was fast. I'm not up on the BSD
> kernel, but in Linux, when using devices that support scatter/gather
> DMA, packets are built as chains of pages. There's a certain amount of
> allocation/copy overhead per page, so you get the best performance when
> you just fill up 2 pages, and get a bit of a dip once you need 3. (This
> might not be the same effect as in BSD, since it's using a constant
> that's not page size...)
I just tried with linux on the macpros (same machines), and with TSO
disabled, do I see a consistent jump of 230Mb/s (9450 -> 9680) for a
normal TCP_STREAM. For a zero-copy TCP_SENDFILE, it seems to actually
hurt performance a little bit (9675 -> 9650). Neither is as pronounced
as the 7000 -> 8000 jump I see on MacOSX.
> If it goes back in, maybe it should have a sysctl? It's not clear this
> optimization is always the right thing to do. There's a trade-off
> between CPU efficiency and network efficiency which is a moving target
> depending on the relative speeds of the CPU and NIC. When it's really
> mattered for me, I've been satisfied just setting the interface MTU to
> get the desired effect.
I'd certainly be OK with a sysctl. I'm afraid that adjusting the MTU
by hand may not work well, when you consider that the TCP (and IP)
header size is not fixed. The common RFC1323 TCP timestamp option,
for example, alters the header size by 12 bytes. This is why I'd
prefer that the mss rounding code go back into the stack.
Drew
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden