Socket recv problems in real-time context : lock or priority inversion issues?
Socket recv problems in real-time context : lock or priority inversion issues?
- Subject: Socket recv problems in real-time context : lock or priority inversion issues?
- From: Stéphane Letz <email@hidden>
- Date: Thu, 19 Mar 2009 09:53:23 +0100
Hi,
We are using UDP sockets to transmit audio in real-time between 2
machines. Basically the "master" machine is synchronized on the audio
interrupt and send/recv audio packets on the "slave" machine. The
"slave" machine is synchronized on incoming packets (that is waiting
for an incoming packet on the socket to start it's audio cycle). The
"slave" machine reception thread is set-up in real-time mode (using
thread_policy_set /THREAD_TIME_CONSTRAINT_POLICY kind of API).
It works quite reliably until we start to load the slave machine (like
starting a big compilation). In this case the RT slave thread does not
"wake up" the the right time, and we have quite big latencies (up to
50 millisecond latency measured on a MacPro Intel 4 cores 10.5.6 when
the audio cycle is supposed to be 5 millisecond). More precisely it
seems the socket recv operation blocks a lot of time.
We configure the socket with SO_SNDBUF/SO_RCVBUF and possibly
SO_RCVTIMEO kind of options. The same code works without problem on
Linux or Solaris.
Using Shark "system trace", we were able to measure abnormal latencies
on the recv function (20 ms in a given test), more precisely in the
"recvfrom_nocancel" one. Using the Shark "Time profile" we see the
following call stack for "recvfrom_nocancel" in the kernel:
0.3% 28.9% mach_kernel recvfrom_nocancel
0.3% 27.5% mach_kernel socketpair
4.0% 24.4% mach_kernel soreceive
0.3% 10.6% mach_kernel sbwait
0.0% 10.0% mach_kernel msleep
0.0% 10.0% mach_kernel uiomove
0.0% 9.7% mach_kernel lck_mtx_sleep_deadline
0.0% 7.7% mach_kernel thread_block
0.0% 7.7% mach_kernel thread_block_reason
7.7% 7.7% mach_kernel ml_set_interrupts_enabled
0.0% 1.4% mach_kernel assert_wait_deadline
0.0% 1.4% mach_kernel wait_queues
1.4% 1.4% mach_kernel ml_set_interrupts_enabled
0.6% 0.6% mach_kernel lck_mtx_lock
0.3% 0.3% mach_kernel thread_block
0.3% 0.3% mach_kernel _mutex_assert
So it may be a lock or priority inversion of some functions used in
kernel mode? Is this a known problem for sockets on OSX? It this trace
enough to do a bug report?
Thanks.
Stephane Letz
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Macnetworkprog mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden