Re: Accessing audio samples from Audio Queue Buffers
Re: Accessing audio samples from Audio Queue Buffers
- Subject: Re: Accessing audio samples from Audio Queue Buffers
- From: Steven Winston <email@hidden>
- Date: Mon, 16 Nov 2009 17:12:27 -0800
Hello KappA,
Congrats! Once you hit the point of dealing with just PCM data you
have full and complete control over what to do next. Professionally I
work at SRS Labs, we do some rather advanced things to PCM data at the
stage you currently have this project at, aimed at making the device
we work on sound as good as we can possibly make it. Some material
that might be of interest to you could be found at:
http://www.aes.org. The digital library should hold a lot of useful
information for some ideas for what you can do next.
In any case, your question was where to go next. Honestly, my opinion
is that's a tremendously large question. There's a ton of processing
things that can be done to PCM data to make the sound "different."
Some make the sound better, some make it worse. I think the first
step is simply to experiment. I think the easiest change of all is to
adjust the gain (i.e. multiply the PCM samples by a scalar that
represents what gain you want to give it, 0.0f for off 1.0f for full
volume).
If you're having problems actually getting this to work, then some
things to look at:
1.) is the PCM data endianess correct for what you're doing?
2.) are you accessing the PCM data correctly?
3.) is it interleaved and being handled correctly per channel?
4.) Are you writing the PCM data back to the buffer in the exact same
way that it was before the changes?
Here's some quick code that I just put together... sorry don't have a
debugger in-front of me at the moment so apologies if my brain missed
something... However, the basic idea is there, break the buffer up
into block sizes (in this case the size of one short for simplicities
sake), get it into the correct channel if it's interleaved, modify,
then place it all back the way it was.
for(int i = 0; i < buffer->mAudioDataByteSize / 2; i+= 2) {
//for interleaved 2 channel audio:
*tempBuf[kLeft] = *(((short*)(buffer->mAudioData)) + i);
*tempBuf[kRight] = *(((short*)(buffer->mAudioData)) + i + 1);
// do some sort of processing here let's add gain:
*tempBuf[kLeft] *= gainChange;
*tempBuf[kRight] *= gainChange;
// now place everything the same as it was
*((((short*)(buffer->mAudioData))) + i) = *tempBuf[kLeft];
*((((shrot*)(buffer->mAudioData))) + i + 1) = *tempBuf[kRight];
}
Thanks,
Steve
On Mon, Nov 16, 2009 at 11:25 AM, KappA <email@hidden> wrote:
> Device: iPhone OS 3.0
> Issue: Applying effects to the Remote IO Audio Unit playback Callback
>
> Hey Guys!
>
> Wow what a journey it's been!!! After leaving this thread back on September
> 9th - I've since then figured out how to successfully implement the Remote
> IO Audio Unit for playback (both local and streaming HTTP audio). Keeping
> tuned to this mailing list has also helped me catch up on bits of useful
> info as well.
>
> At first, I've tried a somewhat hackish method to get things working - where
> I saved the HTTP audio buffers to a temp file, then used the ExtAudioFile
> API's to read it back into memory for playback - but there was glitching :(
>
> What I did to get it working:
> ====================
> 1. Changed my existing AudioQueue implementation to use Audio Queue Offline
> Rendering (as suggested on this list - thanks!)
> 2. After almost giving up, I was able to get things working, after ensuring
> that I allocated the offline buffers first,
> and set the proper ASBD data structures, and request smaller chunks of
> data
> 3. Now I am able to fill offline bytes to my buffer list using the Audio
> Queue, and play it via the Remote IO AU - without glitching!
>
> Now - My challenge is adding effects:
> ===========================
> I have tried the "Phaser" C++ class (http://musicdsp.org/files/phaser.cpp)
> for adding some effects, as well as the EQ mentioned here:
> http://musicdsp.org/archive.php?classid=3#236
>
> But I hear lots of feedback/noise, to the point where it almost drowns out
> the music completely. I am wondering if the settings that I am using to
> implement these are just wrong, or if these even work at all? (I've never
> messed with this stuff before).
>
> Has anyone successfully implemented any of these? If so, any pointers on how
> to use them or if they would even work? Or are there any other C/C++
> routines out there that I can try?
>
> I am obviously a newbie in this arena and I'd love to learn about this
> aspect of things... If there any tutorials one can recommend (C/C++) that I
> can follow, just to get some basic effects going, I'd check it out too.
>
> Any small effect that can make me smile after getting this far would be
> nice. I've even tried multiplying the audio sample by various integers with
> hopes for something cool, but no luck - just static and noise! :(
>
> I feel like I'm so close. If I can't seem to make progress going this route,
> I might just try adding the iPod EQ in the meanwhile (once I figure out how
> :)).
>
> Any help is appreciated to help get some effects going.
>
> Thanks,
> KappA
>
> On Thu, Sep 10, 2009 at 7:42 PM, William Stewart <email@hidden> wrote:
>>
>> one more point of clarity.
>>
>> The software decoders (AAC, MP3) are only available with 3.0 - to decode
>> these on a < 3.0 device, you would need to use the h/w decoder which means
>> you would need to use the AQOffline code.
>>
>> WIth 3.0 you have access to s/w versions of nearly all of the decoders
>> (the exception is HE-AAC which is still only supported with h/w assist).
>> With 3.0 we released an AAC encoder (h/w assist) for some devices.
>>
>> With 3.1 you have access to the h/w versions (encoders and decoders)
>> through the use of ExtAudioFile as well as AudioQueue. The main thing that
>> this gives you is the ability to use ExtAudioFile to write files with AAC
>> encodings (for decoding you could use the s/w decoders already with 3.0)
>>
>> HTH
>>
>> Bill
>>
>>
>> On Sep 10, 2009, at 2:40 PM, Steven Winston wrote:
>>
>>> That might solve a few problems. Thanks Doug; this information should
>>> prove to be most valuable to our application.
>>>
>>> Thanks,
>>> Steve
>>>
>>> On Thu, Sep 10, 2009 at 2:35 PM, Doug Wyatt <email@hidden> wrote:
>>>>
>>>> For some preliminary context: on the phone, the hardware codec tend to
>>>> take
>>>> longer to decode but consume less power. Software codecs tend to execute
>>>> more quickly but are more CPU and power-intensive.
>>>>
>>>> Software codecs are available at any time via AudioConverter, including
>>>> the
>>>> higher-level APIs which use AudioConverter -- AudioQueue and
>>>> ExtAudioFile.
>>>>
>>>> But hardware codecs are only available via AudioQueue. (Exception: as of
>>>> OS
>>>> 3.1, you can use the AAC hardware encoder via ExtAudioFile...)
>>>>
>>>>
>>>> So, if you've been using AudioQueueOfflineRender to decode and
>>>> post-process
>>>> AAC or MP3, you may want to try using AudioConverter to do a software
>>>> decode
>>>> instead.
>>>>
>>>> Doug
>>>>
>>>>
>>>>
>>>> On Sep 10, 2009, at 14:16 , Steven Winston wrote:
>>>>
>>>>> So Doug, This means that we'd be using the 'software' decoders and
>>>>> not the 'hardware' decoders? So there's really no benefit towards not
>>>>> including a software decoder library as it's not really going to the
>>>>> hardware decoder, then providing PCM back to the OS prior to sending
>>>>> it to the DAC?
>>>>>
>>>>> Have a feeling you can't answer this in its' entirety so just a quick
>>>>> explanation about what's available would be most helpful.
>>>>>
>>>>> Thanks,
>>>>> Steve
>>>>>
>>>>> On Thu, Sep 10, 2009 at 2:12 PM, Doug Wyatt <email@hidden> wrote:
>>>>>>
>>>>>> Oops, I was just reminded that we *do* have a software MP3 decoder as
>>>>>> of
>>>>>> 3.0. I don't know what I was thinking.
>>>>>>
>>>>>> sorry for the confusion
>>>>>> Doug
>>>>>>
>>>>>> On Sep 10, 2009, at 13:55 , Doug Wyatt wrote:
>>>>>>
>>>>>>> The new capability in 3.1 is to be able to use the hardware AAC
>>>>>>> *encoder*,
>>>>>>> *only* ... nothing has changed with respect to hardware decoders. We
>>>>>>> do
>>>>>>> have
>>>>>>> some software decoders available as of 3.0, but MP3 isn't one of
>>>>>>> them.
>>>>>>>
>>>>>>> As for the difference in MP3 decode rates, that's interesting. If you
>>>>>>> like, file a bug and attach files which illustrate the problem and
>>>>>>> we'll
>>>>>>> investigate.
>>>>>>>
>>>>>>> thanks,
>>>>>>> Doug
>>>>>>>
>>>>>>> On Sep 9, 2009, at 17:43 , Steven Winston wrote:
>>>>>>>
>>>>>>>> Well in my tests I'm noticing that some audio when encoded with the
>>>>>>>> exact same format are filling the audioqueueofflinerender buffers at
>>>>>>>> different rates than other files encoded with the same format. I
>>>>>>>> didn't investigate too far into this however, it seems like there
>>>>>>>> must
>>>>>>>> be some overhead that prevents the callback happening at a more
>>>>>>>> consistant rate between the files.
>>>>>>>>
>>>>>>>> So for instance if file A is encoded as a MP3 44.2 khz 16 bit 2 chan
>>>>>>>> file and file B is also encoded as a MP3 44.2 khz 16 bit 2 chan file
>>>>>>>> when they are decoding file A might decode at an appreciable rate
>>>>>>>> slower than file B when using AudioQueueOfflineRender. If that
>>>>>>>> problem doesn't exist, in the ExtAudioFile, then I can scrap the
>>>>>>>> external software MP3 decoder in my project.
>>>>>>>>
>>>>>>>> On Wed, Sep 9, 2009 at 5:33 PM, William Stewart<email@hidden>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> On Sep 9, 2009, at 5:28 PM, Steven Winston wrote:
>>>>>>>>>
>>>>>>>>>> That's awesome! However, a question, is this solution going to
>>>>>>>>>> suffer
>>>>>>>>>> from the same different files of the exact same format decode at
>>>>>>>>>> different rates problem?
>>>>>>>>>
>>>>>>>>> "same different" - sorry, but I don't understand the problem you
>>>>>>>>> are
>>>>>>>>> referring too.
>>>>>>>>>
>>>>>>>>>> I'll start digging through 3.1 docs and will
>>>>>>>>>> eagerly await the further details.
>>>>>>>>>>
>>>>>>>>>> On Wed, Sep 9, 2009 at 5:22 PM, William Stewart<email@hidden>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> One of the new things in iPhone OS 3.1 is that you can use
>>>>>>>>>>> ExtAudioFile
>>>>>>>>>>> to
>>>>>>>>>>> read and write compressed data where you may need to use a
>>>>>>>>>>> hardware
>>>>>>>>>>> assisted
>>>>>>>>>>> codec (such as AAC encoding) to do so.
>>>>>>>>>>>
>>>>>>>>>>> Details are forthcoming, including some example code - have a
>>>>>>>>>>> look
>>>>>>>>>>> in
>>>>>>>>>>> the
>>>>>>>>>>> tech notes and release notes for the 3.1 release
>>>>>>>>>>>
>>>>>>>>>>> Bill
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> Do not post admin requests to the list. They will be ignored.
>>> Coreaudio-api mailing list (email@hidden)
>>> Help/Unsubscribe/Update your Subscription:
>>>
>>> This email sent to email@hidden
>>
>> _______________________________________________
>> Do not post admin requests to the list. They will be ignored.
>> Coreaudio-api mailing list (email@hidden)
>> Help/Unsubscribe/Update your Subscription:
>>
>> This email sent to email@hidden
>
>
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden