Re: Simultaneous read() and write() on same TCP/IP socket
Re: Simultaneous read() and write() on same TCP/IP socket
- Subject: Re: Simultaneous read() and write() on same TCP/IP socket
- From: "Quinn \"The Eskimo!\"" <email@hidden>
- Date: Tue, 14 Dec 2010 09:10:03 +0000
On 14 Dec 2010, at 02:37, Jonathon Kuo wrote:
> I'm writing a multithreaded message store-and-forward app using Cocoa and TCP/IP sockets. One quick question: If I have a read() and a write() posted on the same socket from different background threads, is this going to work as I expect it to, basically that the one socket is full-duplex, so I won't be reading what I've written to the socket?
This will work at the socket level, but it's very easy to tie yourself up in knots. Here's a chunk of text I wrote for DevForums that covers the basic issue.
> At the lowest user space level (BSD sockets) it's certainly possible to have two threads simultaneously reading and writing from a socket. However, it's not much fun once you start considering all the edge cases. For example, consider shutdown. What happens when the write thread decides it wants to shut down the connection? How does it get the read thread, which is blocked deep in the kernel, to unblock itself?
>
> Similarly, there are issues with sharing data. In a situation like this you typically have one 'connection' object that stores state that's common to both threads. That connection object must be locked with a mutex. But one thread can't block inside the kernel while holding that mutex because it'll deadlock the other thread. So you have to drop the mutex before blocking. That means you have to a reference count on the connection object to prevent it going away while you're blocked. Fair enough. Now what happens when the other thread changes the state of the shared data such that the blocked thread needs to know about it (for example, the read thread generates a reply that it wants to queue on the write thread's send queue). In a threaded world you traditionally handle this with a condition variable. However, condition variables and sync blocking I/O don't mix. You can't block on a read /and/ wait for the condition variable being signalled. So you have to figure out some other way to wake up the other thread.
>
> Lots of folks try to work around these problems using one of the techniques described below, all of which have their issues:
>
> A. shutting down by closing the socket -- While this actually works (most of the time; don't try it with pseudo TTYs :-) it's subject to nasty race conditions. You can't tell whether your opposite thread is also deciding to shut down the connection, and thus closing the socket out from under you. This makes it very easy to accidentally double close a descriptor which is usually harmless but, if there are other threads opening descriptors, can result in you close their descriptors and messing stuff up royally. This is the sort of thing that makes for /really/ hard to debug threading problems.
>
> B. inter-thread signals -- To paraphrase Herman Göring, "When someone says signals I reach for my revolver." It's incredibly hard to get inter-thread signals to work correctly because of the lack of coordination between sync blocking I/O and <x-man-page://2/sigwait>.
>
> C. secondary descriptors -- You can solve many of these problems by using a secondary descriptor and always blocking using <x-man-page://2/select>. This is illustrated by the SocketCancel sample code.
>
> <http://developer.apple.com/library/mac/#samplecode/SocketCancel>
>
> However, once you start heading down this path you lose most of the advantages that sync blocking I/O had in the first place. Specifically, you can no longer treat socket descriptors like standard descriptors: you have to search through your code and replace all your standard <x-man-page://2/read> calls with special read calls, and jump through even tighter hoops if the code does anything non-trivial with the descriptor (like call select() on it). And this ignores the cost in terms of both descriptors and performance.
If you're still working on the socket-to-socket bridging code, I recommend you look at non-blocking sockets with GCD. That's how I'd tackle this problem on modern Apple OSes.
S+E
--
Quinn "The Eskimo!" <http://www.apple.com/developer/>
Apple Developer Relations, Developer Technical Support, Core OS/Hardware
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Macnetworkprog mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden