Re: UDP Rate
Re: UDP Rate
- Subject: Re: UDP Rate
- From: Justin Walker <email@hidden>
- Date: Sat, 21 Feb 2004 13:31:21 -0800
On Friday, February 20, 2004, at 11:48 PM, Zack Morris wrote:
Hi, I have written a reliable UDP protocol for our games that uses
windowing, resending, ignoring duplicate packets, etc. It's easily
the most
hellish thing I've ever written, I highly recommend against anybody
doing
it, it was harder than opengl, sound double buffering, anything :)
Anyway,
it all works, but I am getting some weird numbers for the throughput.
Here
they are, at a 90% packet success rate:
A couple of things:
- Airport is nominally either 11 or 54 Mbits (that's really rough)
- looks like you are using 100Mbit ethernet, so at best, Airport
will be half the speed
- yes, air is tougher on packets than cable. I don't have any stats
on hand to quantify that, though.
512 bytes+header sized packets:
100bt ethernet:
#packets in window->speed: 16->350k/s, 32->550k/s, 64->750k/s,
128->850k/s
Airport:
#packets in window->speed: 8->110k/sec, 16->80k/s
localhost:
#packets in window->speed: 16->410k/sec, 128->460k/s no loss rate
8192 header+header sized packets:
100bt ethernet:
#packets in window->speed: 8->1.2M/s, 16->1.35M/s
Airport:
#packets in window->speed: 4->135k/s behavior
erratic
localhost:
#packets in window->speed: 8->1.1k/sec at 95%, 16->320k/s at 50%
success
Does anyone know why airport is so slow? Are the packets just
inherently unreliable over radio? Airport should be getting more like
1MB/sec.
Also, is there some setting, like only let 10% of a LAN's traffic
be UDP
packets? These numbers all seem very low to me, certainly lower than
the
theoretical ~10MB/sec for ethernet and who knows how high for
localhost.
I'm not aware of any throttling in the Mac<->Airport<->Mac connection.
If you have some other weirdo device in the path, I can't speak to that.
I also noticed that the program is extremely sensitive to things
like
GetNextEvent() and printing to the console. If I do that very much at
all,
the profiler shows an obscene amount going to the GUI. For instance,
running the program localhost only runs half as fast as running on LAN
because it draws twice as many lines to the screen, even though I am
only
writing a line every 100k! (send+receive, see?)
Yup; using the GUI is going to use a bunch of CPU; that scrolling stuff
takes real cycles. Try logging to a file and viewing it remotely...
FYI I am using nonblocking sockets with an Idle() routine I call
as fast
as the computer can run with GetNextEvent(). I limit it with a timer
that
only allows Idle() to execute 100 times per sec, otherwise it returns.
Anyway, my send and receive loops seem to run ~50 times per sec
because of
missing the heartbeat/main loop stutter, which is perfect for my
needs. I
am going to use 512 byte packets with a window size of 16 packets,
with a
min window send interval of 10ms (normally averages 100ms tho). So
that is
512x16x100 = 819200 bytes/sec. I never see anywhere near this, I can
only
get to 350 k/sec with 512 byte packets.
Here are some specific questions, maybe the gurus can answer at their
leisure:
Gurus don't have leisure :-}
1. Is the sockets stack 64k?
If you mean "the socket's buffer", you can verify what the default is
with 'sysctl':
net.local.stream.recvspace: 8192
net.local.stream.sendspace: 8192
net.local.dgram.recvspace: 4096
net.inet.tcp.sendspace: 32768
net.inet.tcp.recvspace: 32768
net.inet.udp.recvspace: 42080
net.inet.raw.recvspace: 8192
This means that, e.g., for TCP, the default buffer *space* is 32KB,
while for UDP, it is 42KB. The local buffer sizes are trimmed down
some.
You can use the SO_RCVBUF and SO_SNDBUF socket options to modify this
for your process (or sysctl, for the system as a whole. The sysctl
variable "kern.ipc.maxsockbuf" gives the max for each of these settings
(in bytes). Note that the send and receive "space" counts the max that
is permitted in each socket buffer, and does not preallocate real
space. This is complicated a bit by the way the socket code counds
space. Generally, this is based on the size of the mbufs in use, not
the size of "valid data", so, e.g., an 8KB recvspace value does not
mean 8KB of user data; it means 8KB of mbuf space, whether it is in use
or not. It may sound funky, but it does make sense, since it's buffers
that get used up, not parts of buffers.
I can only seem to send about 500k/sec (512
bytes times 128 length window = 64k) about 8x a sec and get no loss
rate.
Same for 8192x8=64k at about 16x a sec. Is this why localhost is
slower
than network, I am running out of memory? Or is there something
inherently
slow with calling sendto() and recvmsg()?
I think you may be running into a variety of issues: using the GUI will
inherently slow other stuff down, and if it is not directly involved in
your task, you should find another way; and you need to experiment with
twiddling the socket buffer space, the size of the buffers your
application is trying to send or fill. See what works best. If you
get a program like 'netperf' (google is your friend), you should be
able to focus on the raw networking issues, to see what the underlying
media will give you "best possible".
2. Is the udp mtu in OS X 8192 bytes? I can't seem to send 16384 byte
packets, or even 16000 bytes. And I thought the mtu for ethernet was
64k.
The MTU for ethernet is 1500 bytes; I'm not sure where you got the 64K
value (on some gigabit links, you can up the MTU to ~9KB, but that has
a host of issues to accompany it). The max UDP datagram is about 64KB,
as I recall, but most systems limit the size of a datagram.
What do you mean "can't seem to send 16384 byte packets"? Knowing such
minor details as error messages/codes helps us to assist in diagnosing
problems.
3. If the packet send success goes over 90% I turn up the speed, if
it goes
below 75%, I turn it down. Are these reasonable numbers? I am afraid
maybe
modems never go above 80% for instance.
What do you mean by "success" (see above)? Remember that UDP does not
guarantee delivery, and it won't tell the sender if it (or the driver)
has to drop packets.
4. Is there any benefit to threaded sockets?
Do you mean using the same socket in multiple threads? No. The data
all goes to the same place, and since there is no throttling of the
kind you are worried about in the system, this will just make things
worse.
I am going to be calling
Idle() at least 30 fps in my game's loop, so I can't see how it would
go
faster. I was thinking of making a thread with select() that only
runs when
data comes in, or I would signal the thread to run by sending data to
it on
a local socket (is this a pipe???). My code is about 90% interrupt
safe, as
I was writing it to run on 9 too, but if I can avoid
threads/interrupts,
that would be great as they are an evil icky can of disgusting worms.
For GUI things like GetNextHooHah, you will need to talk to the guy
over in that line :-}.
5. Are my numbers even close to right? Have any of you programmed a
UDP
game before? Is UDP just for signalling or something? My bandwidth
should
be 10 times higher than it is, and I don't know if my main loop just
isn't
running fast enough, or if the OS/network is limiting me. Okay I have
infinite questions so I will stop now.
TCP works well in the face of things like congestion and other "packet
loss" issues because it has a lot of built-in self-tuning algorithms to
keep data flowing. UDP is like unleashing the kids at the playground.
If it's coherence you want, you either have to use TCP, or reinvent its
wheels (or, at least, those wheels that are of use to you).
Whether your numbers make sense is not easy to answer without a whole
lot more understanding of the app, the environment, and the network.
You might want to invest in Stevens's Unix Network Programming, V1, 2nd
ed. I don't know that it directly addresses your issues, but it
contains a wealth of stuff that surrounds those issues.
Regards,
Justin
--
Justin C. Walker, Curmudgeon-At-Large *
Institute for General Semantics | Men are from Earth.
| Women are from Earth.
| Deal with it.
*--------------------------------------*-------------------------------*
_______________________________________________
macnetworkprog mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/macnetworkprog
Do not post admin requests to the list. They will be ignored.
| References: | |
| | >UDP Rate (From: Zack Morris <email@hidden>) |