Re: NSFileHandle readInBackground vs threading?
Re: NSFileHandle readInBackground vs threading?
- Subject: Re: NSFileHandle readInBackground vs threading?
- From: Andreas Grosam <email@hidden>
- Date: Wed, 09 Nov 2011 10:07:20 +0100
On Nov 8, 2011, at 5:54 PM, Alexander Bokovikov wrote:
> I have a need to read some data from a local socket, which serves for IPC. And data may come very quickly, so (AFAIU) inner socket buffer might overflow, so a portion of data might be lost.
This has been answered by previous posters already - but lets assume in your current approach data may possibly be lost when you are not able to consume them fast enough.
> I don't see a way how to define an inner system buffer size, so the only I can is to do my best to read from the socket quickly enough. The problem is that I need yet to process the incoming data, not only to read them. Now I'm doing the next:
>
> - (void) readPacket:(NSNotification *)aNotification {
> [packetBuffer appendData:[[aNotification userInfo] objectForKey:NSFileHandleNotificationDataItem]];
> [sockFileHandle readInBackgroundAndNotifyForModes:modesArray];
> [self processPacket];
> }
The problem here is, that you processes the data on the same thread where you consume it. This may cause the producer to block, and as a result, to discard new or old data eventually if some internal buffer size reach a limit.
You may try to schedule the processing of the data on a different thread. For instance, you can use a dispatch queue where you asynchronously schedule the processing of the data:
#include <dispatch/dispatch.h>
- (void) readPacket:(NSNotification *)aNotification {
NSData* data = [[aNotification userInfo] objectForKey:NSFileHandleNotificationDataItem];
...
dispatch_async(myProcessDispatchQueue, ^{
[self processData:data];
};
}
myProcessDispatchQueue (dispatch_queue_t) shall be a serial queue, e.g. in your init method:
myProcessDispatchQueue = dispatch_queue_create(
"com.mycompany.myapp.data_processing"
DISPATCH_QUEUE_SERIAL);
One important thing to note here:
Since the second approach uses different threads which may possibly access the same object concurrently, we need to consider concurrency and possibly race conditions. Here, the local NSData object 'data' is not subject to race conditions. It is further assumed that -processData: will not violate these rules, that is, it shall not access ivars or globals which possibly could be accessed in a different thread as well without synchronization.
Possible issues that may arise here:
Individual data buffers will be generated much faster than they can be processed. This will cause the dispatch queue to queue up a lot of data buffers and tie up memory. There are solutions to avoid this, but this is more elaborated and requires a concurrent accessible queue which can block producers and consumers.
>
> where packetBuffer is the storage for incoming data and processPacket is where data are processed.
>
> My question is:
>
> Isn't it better to do it in this way:
>
> - (void) readPacket:(NSNotification *)aNotification {
> [packetBuffer appendData:[[aNotification userInfo] objectForKey:NSFileHandleNotificationDataItem]];
> [sockFileHandle readInBackgroundAndNotifyForModes:modesArray];
> [self performSelector:@selector(processPacket) withObject:nil afterDelay:0];
> }
While this might work, this is likely not very efficient. This implicitly assumes, that for every readPacket: invocation, a corresponding processPacket method will be queued in the main thread and then *be executed in strict order* (if not, other issues may arise). If this actually works (because enforced by the run loop), this would result effectively in the same as your current approach above.
>
> Or is the multithreaded processing the only (or at least much better) solution here?
I guess yes.
> If yes, then may I be sure that NSData appendData (see above) will never relocate the initial portion of the data buffer, but only will add new data to the end of buffer?
In other words, you are asking if the addresses of the bytes of the internal data remain stable if you append data to the NSMutableData object? When imagine how difficult this would be - I would assume, NO it isn't.
> My data processing routine looks like the next:
> ................
> len = [packetBuffer length];
> ptr = [packetBuffer bytes];
> while (len >= MIN_PACKET_SZ) {
> /// doing something with data pointed by ptr->
> ptr += _some_value;
> len -= _some_value;
> }
> ................
>
> So, I need be sure that once reading the ptr, then increasing it, step by step up to the len value, I'll always have valid data despite of how many times append data will be called in another thread. What about this?
You'll likely need an approach, say a method that takes an NSData object that contains partial input, then process it, possibly safes internal state and returns. And this method can be invoked continuously to consume another NSData object in order to process the next partial input (taking the current state into account) and do this until the last buffer has been processed.
The method should be able to be invoked on a different thread, see above.
Regards
Andreas
>
> Thanks in advance.
>
> -Alex
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden