Appleshare performance over WAN with latency, quantum size, read sizes, number of outsanding requests, differences between 10.5.8 and 10.6.4.
Appleshare performance over WAN with latency, quantum size, read sizes, number of outsanding requests, differences between 10.5.8 and 10.6.4.
- Subject: Appleshare performance over WAN with latency, quantum size, read sizes, number of outsanding requests, differences between 10.5.8 and 10.6.4.
- From: Jan Snydr-Michal <email@hidden>
- Date: Thu, 15 Jul 2010 17:50:53 +0200
Hi Folks,
I have a couple of questions regarding the AppleShare client
implementation in 10.5.8 and 10.6.4 related to running across long fat
networks (WANs with 100 MBit or greater Bandwidth with a 45
millisecond roundtrip latency.) Questions are on the bottom,
background first. I'm hoping that someone in the know with access to
the client source may be on this list (Brad S or Leland W still
working on the client?)
I'm working with a large publishing customer in the UK that is
reporting that they are getting 9 MByte/second transfers over their
WAN using ftp, but doing a Finder copy of a single 100 MByte file they
were seeing 1MByte/second transfers, this using a 10.5.8 client going
to commercial AFP server on Linux with a 384 KByte quantum.
Looking at the packet traces I discovered that the server offered a
384 KByte quantum in the AFP opensession reply, but the client was
issueing ReadExt and WriteExt in 32 KByte chunks for this finder
copy. After a bit of investigation I found this article http://support.apple.com/kb/TA24270
where it is explained that the appleshare client can go into a
"WAN" mode, and AppleShare Client when in "WAN" mode would use a
different and reduced quantum. This document, as well looking at
packet traces using 10.5.8 as a client, show that:
With a 10.5.8 client:
a) on LAN (latency less than afp_wan_threshold) the AFP client breaks
up reads/writes into 128 KByte operations
b) on a WAN (latency more than afp_wan_threshold), the AFP client
breaks up reads/writes into afp_wan_quantum sizes, the default being
32 KBytes. Setting afp_wan_quantum greater than 128 KBytes results in
the client using a 32 KByte quantum.
c) The client does up do 3 concurrent outstanding requests.
c) statfs returns f_iosize of 512 KBytes on the AFP volume.
This is a large problem on "long fat networks", as not enough data is
"in flight" to effectively use the available bandwidth, and lots of
the time during a trace is spent where the server is waiting for the
next request from the client to reach it. The Bandwidth delay product
of a network with 100 Mbit/sec bandwidth x 0.050 seconds latency per
round trip = 5 MBit = 610 KBytes. Thus, to effectively use the
bandwidth given the delay, we need to have 610 KBytes in flight. With
3 outstanding requests of 32 KByte each (the default 10.5.8 client
setup), we are only going to effectively use 1/6 of the available
bandwidth, after tuning the afp_wan_xxx defaults we can get the client
to issue up to 3 128KByte requests resulting in being able to up to
1/2 of the available bandwidth. Things look much worse for a gigabit
wan. The rationale for reducing the amount of data "in flight" on
networks with high latency escapes me, as for those networks one needs
to increase the amount of data in flight to effectively use the
available bandwidth.
Looking at packet traces on 10.6.4 things look very different!
On 10.6.4 clients I see (based on packet traces to a OS X built in
filesharing AFP server running on 10.5.8, which offers a 256 KByte
quantum in the opensession):
a) on LAN the AFP client issues ReadExt in 256 KByte sizes.
b) on a WAN with 40 millliseconds delay (and 16 Mbit bandwidth) to a
server with a 128 KByte quantum I see interesting things!
-the size of requests varies between 32 KBytes and 128 KBytes (for a
single Finder copy of a large file!)
-the number of outstanding (concurrent) requests from the AFP client
varies, when using smaller reads more are issued concurrently.
c) statfs returns f_iosize of 512 KBytes on the AFP volume
It appears, that based on network performance data (and NOT just
latency?), the AFP client in 10.6.4 tunes itself and does different
sized reads and different number of outstanding requests that takes
into account network bandwidth and latency.
Here are my questions:
1) Is there any documentation anywhere (or can someone explain the
meaning, defaults, and value range of), the com.apple.AppleShareClient
preferences in 10.6.4:
-afp_maxQuantumSize
-afp_maxQuantumNbr
-afp_maxSingleIOToleranceMSecs
-afp_maxIOToleranceMSecs
(and the corresponding mins).
I'm particularly interested in the defaults, and max values!
2) Can someone explain the logic that is used in 10.6.4 AFP client to
determine used quantum size and number of outstanding requests? I'm
assuming this involves the answers to question (1). Does the client
issue requests so that number of outstanding requests * request size =
server quantum size (kind of looks that way).
3) Does this mean that 10.6.4 no longer uses the afp_wan_threshold and
afp_wan_quantum ("the madness").
4) Is there any way to get the AFP client in 10.5.8 use a quantum
larger than 128 KBytes on 10.5.8 (I'd consider editing a binary if
necessary).
Thanks for any info!
greetings,
-Jan-
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden