Re: How to throttle rate of NSInputStream?
Re: How to throttle rate of NSInputStream?
- Subject: Re: How to throttle rate of NSInputStream?
- From: Andreas Grosam <email@hidden>
- Date: Tue, 27 Mar 2012 20:22:29 +0200
On Mar 27, 2012, at 1:13 AM, Jens Alfke wrote:
> [I posted this to macnetworkprog earlier today, but 3 hours later it hasn’t shown up yet, so I’m trying here too.]
>
> I’m using an async NSInputStream to read from a TCP socket. Given a good connection, the data comes in faster than an iOS device can parse it (it’s a stream of small JSON docs, one per line.)
You may look at JPJson framework available on GitHub <https://github.com/couchdeveloper/JPJson>.
The class JPAsyncParser solves this problem. Please examine the "NSURLConnectionDownload" example which shows how to download and parse simultaneously, without stressing memory. It also shows how the parser integrates nicely with NSURLConnection. There is also AppleDoc documentation.
Simultaneous downloading and parsing is a really performance boost. I experienced, that when downloading larger documents, the parser is already finished when the last byte of the document arrived. No temporary file needed - and a really scaleable solution.
Basically, the parser (JPAsyncParser) runs on a secondary thread and parses one or more JSON documents per connection. A client will be "notified" about a complete JSON representation or errors through a Blocks API.
The size of the document containing one or more JSON documents can be unlimited. The parser also does not queue more than one NSData object which will be received in a NSURLConnection delegate method - that is, the memory is bounded. This is accomplished through blocking the thread where the connection delegates will be scheduled.
You can control whether you process the JSON representations *synchronously* with the JSON parser (recommend) or asynchronously (which possibly causes GCD to queue many JSON objects).
For a reason, it avoids using NSStreams.
Caution: the state is still Beta - but all major tests pass.
Have fun!
Regards
Andreas
> I’m performance-tuning my code, and finding that NSInputStream is shoving all of the data down my throat at once, without giving the runloop a chance to process any other input sources. This means that my thread is blocked, and other tasks scheduled on it don’t get a chance to run, until I’ve read the whole feed, which takes over a minute. (And there’s also the fact that CFNetwork must be buffering megabytes of data from the socket that my code hasn’t read yet.)
>
> This seems to hinge on the way the stream calls my event handler. When I get the NSStreamEventHasBytesAvailable, I only read 8k bytes of data at a time, then return. What seems to happen is that, if that didn’t consume all of the available data, the stream will keep sending me the event in a tight loop without exiting back to the runloop in between.
>
> What can I do about this? Ideally I’d like the stream to hold off on reading more from the socket until my code finishes processing the buffer. As a second choice, I’d like it to at least return the runloop more often so my other tasks get time to run.
>
> —Jens
>
>
> _______________________________________________
>
> Cocoa-dev mailing list (email@hidden)
>
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
>
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden