AudioServerPlugIn — Mix operations confusion
AudioServerPlugIn — Mix operations confusion
- Subject: AudioServerPlugIn — Mix operations confusion
- From: Seth Willits <email@hidden>
- Date: Thu, 25 Jun 2015 18:20:42 -0700
As far as I can see there's no documentation on AudioServerPlugIn besides the headers, and the comments on the various mix operations are lacking. I know this is a lot to read, but hopefully my questions and the answers to them will help others in the future too.
I have a couple of questions near the end. Please correct me if I'm wrong at any point.
~~~~
When talking about the operations which get audio from clients *into* my device, the relevant operations are kAudioServerPlugInIOOperationMixOutput, ProcessMix, ConvertMix, and WriteMix.
MixOutput works just as I thought it would. The main buffer in MixOutput has the audio (in the canonical format correct? - despite not explicitly saying so) for a single client only, requiring me to copy the audio out of that buffer, and mix it into my device's ring buffer, converting if necessary. Using this operation, if I only handle IO operations coming from a single particular client, then I get a clean audio signal from just that one client.
The one thing that's confusing is why does it say "this operation always happens in-place" if the entire point of the operation is to copy the audio out of the main buffer and into my device? My understanding of "in-place" means that some audio is put *into* the main buffer, and yet I should only be copying *out* of it. I suppose it's simply trying to convey that the seconday buffer is unnecessary?
~~~~
Now, for the other operations: ProcessMix, ConvertMix, and WriteMix. It seems to me that these can only used if you don't say you do MixOutput (in WillDoIOOperation). Without using MixOutput, WriteMix is required, but ProcessMix and ConvertMix are both optional?
My understanding of how they work together is that the output of ProcessMix is handed into ConvertMix, and the output of ConvertMix is handed into WriteMix, as such:
ProcessMix:
Input: The main buffer contains in canonical format, a mix of all clients' audio.
Output: processed (volume, gain, filters, whatever?) mix of all clients' audio, into either the main buffer or secondary (depending on WillDo's return value for "in-place"), canonical format
ConvertMix:
Input: the output of the ProcessMix operation, in the main buffer, still canonical format.
Output: Simply converted input into the device's native format. Put into either the main buffer or secondary buffer, (again depending on WillDo's in-place value)
WriteMix:
Input: The output of ConvertMix, which is the fully mixed, device-native format audio.
Output: none. (Copying into my device's ring buffer.)
Only WriteMix is required (according to the header), so I imagine it's possible that I could not implement/support ConvertMix, and Core Audio would do the conversion for me, but still let me use ProcessMix and WriteMix?
~~~~
Now, assuming the above is all correct, then I'm really confused about WriteMix's behavior because according to the above (based on my interpretation of the header comments), during a WriteMix operation, the main buffer passed to DoIOOperation contains the fully mixed audio from all of the various clients. So if there are two applications sending audio to my device, I expect to get once DoIOOperation/WriteMix call per sample time range, where the main buffer contains the mix of the two apps' audio, in my device's native format.
Instead, DoIOOperation/WriteMix is called *for each client*, and they have overlapping sample time ranges. For example (sample times are on the end:)
WriteMix: Client 205 -- 2091040.000000
WriteMix: Client 348 -- 2091500.000000
WriteMix: Client 205 -- 2091552.000000
WriteMix: Client 348 -- 2092012.000000
Notice that they completely overlap, so that I always have two WriteMix calls for each sample. This doesn't make any sense to me. I'd expect just one, and the client id for the operation to be 0 since the buffer contains a full mix of multiple clients.
What's really odd is if I only copy the audio from the WriteMix operations that are "from" just one of these clients (either one), it sounds garbled and terrible. If I copy the audio from both (despite the sample times completely overlapping), it sounds correct.
And a second thing, why when only recording from my device (which is both an input and an output device) does WriteMix happen with the client id matching that of the client which is recording from my device (aka ReadInput is happening)? (The main io buffer during these WriteMix calls is all zeros.)
~~~~
I know that's a lot to digest. Hopefully someone can help me out. :-)
--
Seth Willits
--
Seth Willits
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden