Re: dispatch_async Performance Issues
Re: dispatch_async Performance Issues
- Subject: Re: dispatch_async Performance Issues
- From: Andreas Grosam <email@hidden>
- Date: Tue, 28 Jun 2011 19:22:54 +0200
On Jun 28, 2011, at 4:07 PM, Jonathan Taylor wrote:
> ... and more importantly I do not believe your code makes correct use of dispatch_group_async. Your first call adds a producer block to the global concurrent queue, which may start executing immediately. At some time soon afterwards you add a consumer block to the global concurrent queue, which again may start executing immediately - while the producer block is still in the process of "producing". As far as I can see the code appears to have a number of major issues:
>
> 1. Theoretically your producer block may not have produced anything by the time your consumer block executes
The consumer thread may block until a buffer is available. This happens in member function consume() of the buffer list.
>
> 2. What happens if your consumer block has processed all the currently-added blocks before the producer block has added the eof marker?
The consumer block loops until a buffer is available, or until an timeout occurs. This happens in member consume().
An empty buffer signals EOF.
>
> 3. You have only one single consumer block, which means you will only have a single consumer "thread" running under Grand Central Dispatch, which rather defeats the purpose of using it at all! I think you may not have fully understood how GCD is intended to be used...
I know this. But there is no easy way around, due to this: the buffers represent a contiguous stream of data. The consumer thread is used to execute a parser. The parser can not simply stop and then continue with a new chunk of data. The parser's state is represented in the execution stack. So, the parser may block when it iterates over the data buffers and thus safes its state that way.
I know that this is not the optimal way to use GCD. But at least I get semaphores which do not trap into the kernel that often, and thread switching shall be faster as well.
The original goals for this approach is:
Reduce memory foot print. The buffers list maximum number of buffers is limited.
Increase overall performance due to simultaneously downloading data and parsing.
Scaleability . No need to safe the stream to an intermediate file on disk before parsing.
Avoid copying of data completely.
>
> 4. Are you sure that your mysterious and rather complicated-seeming CFDataConstBuffers class
:)
> is sufficiently threadsafe for your producer to be adding to it at the same time as your consumer block is iterating through it? I am 99% sure that the "explicit" consumer code you wrote in ConcurrentProduceConsume is not threadsafe.
I hope it is. But there may be subtle caveats which can in certain circumstances reduce performance due to frequent context switches. I would avoid it if I would know IF and when this can happen ;)
The source is available (see link previous mail). Would you mind taking a look at it? But be warned: this is C++ code.
Thanks in advance!
Andreas
>
> Jonny.
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden