Re: A NSOutputStream vs. native socket question
Re: A NSOutputStream vs. native socket question
- Subject: Re: A NSOutputStream vs. native socket question
- From: Motti Shneor <email@hidden>
- Date: Mon, 02 Dec 2013 22:18:54 +0200
Hi, following my previous discussion I wrote the following method to extract the native socket handle from a given stream (NSInputStream* or NSOutputStream*).
Method works well, but the application crashes few seconds afterwards, claiming a spurious release of an already-released CFData object - I found to be the Native handle property I extract from the stream.
Documentation says I own the CF object returned by CFWriteStreamCopyProperty/CFReadStreamCopyProperty, and it my responsibility to release it - following CF "Create/Copy" method name rule.
Can someone tell me what's wrong here? I can't find my fault.
Hints:
* Crash is consistent.
* If I comment-out the ::CFRelease(nativeSocketProperty), all works fine, but I can't understand why I don't have a leak.
* The method is called once, in the stream event handler, in the EventOpen event.
- (BOOL) retrieveNativeSocketHandleForStream:(NSStream*)stream {
BOOL result = NO;
::CFDataRef nativeSocketProperty = NULL;
::CFSocketNativeHandle temp = NULL;
do {
if (stream==nil || (stream!=_outputStream && stream!=_inputStream))
break;
// Read the native socket property from the stream.
if (stream==_outputStream)
nativeSocketProperty = (::CFDataRef)::CFWriteStreamCopyProperty((::CFWriteStreamRef)stream, kCFStreamPropertySocketNativeHandle);
else
nativeSocketProperty = (::CFDataRef)::CFReadStreamCopyProperty((::CFReadStreamRef)stream, kCFStreamPropertySocketNativeHandle);
if(nativeSocketProperty == NULL)
break;
::CFDataGetBytes(nativeSocketProperty, ::CFRangeMake(0, sizeof(::CFSocketNativeHandle)), (UInt8*)&temp); // Read the actual handle (file descriptor) as int from the property retrieved. can't fail.
if (stream==_outputStream)
self.outputNativeSocketHandle = temp;
else
self.inputNativeSocketHandle = temp;
::CFRelease(nativeSocketProperty); // <----- is this needed? I crash when it is done, and I don't crash if I comment this out.
result = YES;
} while (false);
return result;
}
On 15 בנוב 2013, at 01:14, Josh Graessley <email@hidden> wrote:
>
> On Nov 14, 2013, at 12:58 PM, Motti Shneor <email@hidden> wrote:
>
>>
>> On 14 בנוב 2013, at 21:10, Josh Graessley wrote:
>>
>>>
>>> On Nov 14, 2013, at 10:53 AM, Motti Shneor <email@hidden> wrote:
>>>
>>>> Hi Everyone. First post to this group.
>>>>
>>>> I'm controlling the output stream created by
>>>>
>>>> ::CFStreamCreatePairWithSocketToHost(kCFAllocatorDefault, (CFStringRef)hostAddress, portNum, &readStream, &writeStream);
>>>> ::CFWriteStreamSetProperty(writeStream, kCFStreamPropertyShouldCloseNativeSocket, kCFBooleanTrue);
>>>> ::CFReadStreamSetProperty(readStream, kCFStreamPropertyShouldCloseNativeSocket, kCFBooleanTrue);
>>>>
>>>> and later bridged to NSInputStream and NSOutputStream instances,
>>>> via its internal native socket, using the posix getsockopt and setsockopt calls.
>>>>
>>>> The properties I need to work with are the read (input) and write (output) buffer sizes, and the write (output) socket's buffer pending content size.
>>>>
>>>> Things like this...
>>>> oss = getsockopt(socket,SOL_SOCKET,SO_RCVBUF,&value,&valueSize);
>>>> oss = setsockopt(socket,SOL_SOCKET,SO_SNDBUF, &value, sizeof(value);
>>>> oss = getsockopt(socket,SOL_SOCKET,SO_NWRITE, &value, &valueSize);
>>>>
>>>> etc.
>>>>
>>>> To retrieve the native socket I'm using code like this:
>>>>
>>>> // Getting the amount of bytes in the output socket buffer which are waiting to be sent
>>>> - (int) getSocketOutputBufferPendingSize:(uint32_t*)bufSize {
>>>> int oss = -1;
>>>> CFDataRef data = NULL;
>>>> do {
>>>> if (_outputStream==nil)
>>>> break;
>>>>
>>>> // retrieve native socket handle from output stream.
>>>> if((data = (CFDataRef)::CFWriteStreamCopyProperty((CFWriteStreamRef)_outputStream, kCFStreamPropertySocketNativeHandle))== NULL)
>>>> break;
>>>>
>>>> CFSocketNativeHandle socket;
>>>> CFDataGetBytes(data, CFRangeMake(0, sizeof(CFSocketNativeHandle)), (UInt8*)&socket);
>>>> .
>>>> .
>>>> . // --- here I work with the native socket properties.
>>>> .
>>>> } while (false);
>>>>
>>>> if(data!= NULL)
>>>> ::CFRelease(data);
>>>> return oss;
>>>> }
>>>>
>>>> My questions are:
>>>>
>>>> 1. What is the lifespan of the native socket handle?
>>>> 2. Can I retrieve it once, when the socket is created or opened the first time, then use it until socket is closed, or should I retrieve it from the stream like this, every time I want to use it?
>>>> 3. What is the cost of the above lines that retrieve the native socket handle?
>>>
>>> I hate to answer a question with another question, but…
>>>
>>> Why are you modifying SO_SNDBUF? Modifying SO_SNDBUF and SO_RCVBUF means the system can’t dynamically adjust these values based on network conditions. If you’re hand tuning these values, that might have a small improvement in performance in one network environment but it is likely to have a negative performance impact in other environments.
>>>
>>> Fetching the native file descriptor out of the socket stream does work but it doesn’t come without cost.
>>>
>>> -josh
>>>
>>
>>
>> Hi Josh. I will answer your question, but could I please ask you to also provide some answer to mine?
>
> Sure:
>> 1. What is the lifespan of the native socket handle?
>
> The native socket handle should exist until the socket streams are released.
>
>> 2. Can I retrieve it once, when the socket is created or opened the first time, then use it until socket is closed, or should I retrieve it from the stream like this, every time I want to use it?
>
> To the best of my knowledge, you should be able to retrieve it once.
>
>> 3. What is the cost of the above lines that retrieve the native socket handle?
>
> It is not terribly expensive, but it’s better to cache the results and trust it won’t go away until the socket streams are closed.
>
> A potentially safer option is to get the native socket handle once and call dup on the socket you get back. Your dupd file descriptor will be yours to close when you see fit. The socket itself won’t be closed until all file descriptors that point to it are closed.
>
>> In the organization I'm working for (as a contractor), we're using an old, (and dare I say wracked) "socket" implementation which maintains its own message queue on a cyclic buffer, from which it draws data and writes it to the NSOutputStream's socket.
>>
>> This socket's design and most of its implementation are taken from an even-older C++ Windows (sigh) implementation over a windows native socket who is opaque and provides no information neither about the size of its buffers, nor of their current contents size.
>>
>> Under those historical windows-imposed considerations, the only way to have reasonable bandwidth measurements (how much was pushed through the network in the last second, and what is the "delay" of data in "our side" of the server connection) is by reducing the size of the socket's internal (opaque) buffers, in comparison with the external socket queue, and do the measurements on OUR queue.
>>
>> When the system socket's internal buffers are small and there is a network bandwidth problem --- we know it early, by failing to write new data into the system socket.
>
> The downside to this approach is that the performance problem could be the small socket buffer sizes. Both your send and receive buffers need to be at least the size of the bandwidth delay product (bandwidth multiplied by the round trip time). On a lower latency network, you won’t see a big performance hit for a small buffer, but when you move to a high latency network, the small buffer could become the limiting factor.
>
>> OF COURSE THIS IS A BAD IMPLEMENTATION.
>>
>> However, I'm a contractor, hired to debug old code, and I can only recommend on enhancing (actually deleting) this old socket code. This is nontrivial because this socket code is doing much more than what I said, and is tightly bound to the specific protocol running between the server and client. It also implements non-standard encryption, nonstandard proxy interactions, and most important -- same socket code runs on server (Windows) and all sorts of clients. In time, I may be called to replace it with better implementation.
>>
>> I know of a good way, using the SO_NWRITE property, to provide good network bandwidth measurements WITHOUT altering the native socket buffer sizes, but I will need to read this property pretty often --- hence my question. What can I do once-per-native-socket and what do I need to do regularly?
>
> Using SO_NWRITE is pretty fast. It’s a trip to the kernel, so I wouldn’t call it every millisecond, but if you’re calling it less often, it shouldn’t show up as a hot spot (but it’s always good to profile the code to confirm).
>
> Getting the send and receive buffer sizes and using SO_NWRITE and SO_NREAD to determine how much data is in the send and receive buffers is a good approach. If you can avoid setting the size of those buffers, you’re likely to see better network performance.
>
> -josh
>
Motti Shneor
e-mail: email@hidden
phone: +972-8-9267730
mobile: +972-54-3136621
----------------------------------------
Ceterum censeo Microsoftinem delendam esse
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Macnetworkprog mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden