So, something I've been wondering about for a while. Prompted by something Benjamin Ludwig wrote this week about wanting to process audio data in network streaming app:
On Oct 4, 2012, at 6:33 AM, Benjamin | Freunde der Interaktion <email@hidden> wrote:
In the moment I'm using Matt Galagher's AudioStreamer and have access to the mp3 samples
(or aac). He's using the Audio Toolbox Framework (Audio Queue Services, Audio File Stream Services,…).
So here's the thing. I've seen a LOT of use of Matt's AudioStreamer project < https://github.com/mattgallagher/AudioStreamer> out in the wild. One tell-tale sign is how his AudioQueueOutputCallback function throws things over to an Obj-C method called handleBufferCompleteForQueue:buffer:. Googling for " handleBufferCompleteForQueue" gets me 109 hits, and that's just of the stuff that's public. Lots of people are using this internally.
And that's fine… this is battle-tested code that Matt's put plenty of work into, which means that you can pick it up and use it with confidence.
But here's the thing about his implementation: he reuses a fixed set of AudioQueueBuffers. The number is set with a constant, kNumAQBufs, and the pool is created in a createQueue method, with each one sized to the value of the Audio File Stream property kAudioFileStreamProperty_PacketSizeUpperBound, kAudioFileStreamProperty_MaximumPacketSize, or just a constant kAQDefaultBufSize (defaulting to 2048 bytes).
By using these buffers, he has to do some careful work in handleAudioPackets:numberBytes:numberPackets:packetDescriptions: (called by his AudioFileStream_PacketsProc function) to fill buffers to the best of his ability: he has a current buffer (fillBuf, which is a local variable referencing audioQueueBuffer[fillBufferIndex]) that is progressively filled with each call into this method, enqueued only when the newly-received packet data would overflow the AQ buffer. This means keeping precise track of offsets into the current buffer to make this progressive fill work.
And it does work. But it seems like a lot of hassle, more than I feel like teaching.
So lately -- in my Core Audio classes, and in a web radio project I'm working on -- I took a different tack: disposable AQ buffers.
In other words, when I get the packets callback from the audio file stream, I create an audio queue buffer of the exact size I need for that packet data, and enqueue it. When the audio queue callback tells me the buffer has been used, I just dispose the buffer.
But seriously, here's the audio file packets callback and audio queue output callback:
void streamPacketsProc ( void *inClientData, UInt32 inNumberBytes, UInt32 inNumberPackets, const void *inInputData, AudioStreamPacketDescription *inPacketDescriptions ) { NSLog (@"%ld packets", inNumberPackets); CCFWebRadioPlayer *player = (__bridge CCFWebRadioPlayer*) inClientData;
AudioQueueBufferRef buffer; CheckError(AudioQueueAllocateBuffer(player.audioQueue, inNumberBytes, &buffer), "couldn't allocate new queue buffer"); buffer->mAudioDataByteSize = inNumberBytes; memcpy(buffer->mAudioData, inInputData, inNumberBytes); CheckError(AudioQueueEnqueueBuffer(player.audioQueue, buffer, inNumberPackets, inPacketDescriptions), "couldn't enqueue buffer");
if (((totalPacketsReceived+=inNumberPackets) > 100) && (!queueStarted)) { CheckError(AudioQueueStart(player.audioQueue, NULL), "couldn't start audio queue"); queueStarted = YES; NSLog (@"started audio queue"); } }
#pragma mark audio queue stuff void queuePlaybackProc (void *inUserData, AudioQueueRef inAQ, AudioQueueBufferRef inBuffer ) { CCFWebRadioPlayer *player = (__bridge CCFWebRadioPlayer*) inUserData; CheckError(AudioQueueFreeBuffer(player.audioQueue, inBuffer), "couldn't free old buffer"); }
About 40 lines. Easy to follow, easy to teach. And has held up with the streams we've played with in class. Clearly, the code could be made fancier and more battle-ready (for starters, it does not attempt the needed heroics to tease out the stream type and instead just assumes MP3).
Is one better than the other? Seems like the tradeoff is creating and destroying more buffers and potentially having lots of little ones in the queue (my way) versus having more memory tied up in idle buffers than you really need (Matt's way). In 2012, even on the device, I'd bet that neither of those things matters all that much.
I feel like if Matt is working so hard to reuse buffers, he likely has a good reason to do so. But it could just be that he coded by analogy to the early iPhone Core Audio file-playback sample code, which reused a queue of three buffers.
Anyways, if there's a good reason to not do this with single-use buffers (CPU expense, increased likelihood of queue starvation, etc.), I'd love to hear it, so I could at least make it a talking point in class, or back out of this approach if I'm sending people down the wrong path.
Thanks!
--Chris
PS: FWIW, in my experience, most of the web radio streams I hit send a bunch of packets in the first callback, which then quickly slows to a trickle. If I log the number of packets I'm getting, I'll get like 80, 20, 1, 1, 1, 2, 1, 1… That's why for class, I found that it generally worked to just start the output queue the first time we enqueue anything to it, since we usually get a bunch of packets on the first callback (again, it would be safer to count how many packets we'd enqueued before starting the queue).
|