Long post with many clarifications and corrections follows.
OK, so I started this whole mess with my CocoaConf demo code of AUNewTimePitch and AudioQueueProcessingTap. This discussion has basically forked at least three ways that I know of:
1. My sample code stopped working on iOS 6.1 (simulator and device) due to an internal render callback being called once and never again
2. The AUNewTime pitch is reported to introduce distortion on the device
3. AUNewTime pitch is an AUConverter (not an AUEffect) and may request more samples of input than it produces as output, and therefore is inappropriate to use in "real time" scenario like AQTap
Taking these in order:
1. Render callback doesn't work on iOS 6.1
I submitted this as Radar 13126776 ("Regression: Audio Unit render callback called once and never again"), also posted to OpenRadar at http://openradar.appspot.com/radar?id=2641401. I opened a DTS incident, and Apple helped figure out that my code was simply bad: an AudioTimeStamp used in my AudioUnitRender() calls was not being incremented with the number of new frames on each pass.
Fixing this got the demo working again on 6.1, but only in the left channel. It turns out that the topography has changed: in 6.0, I seem to get interleaved audio in the AQTap callback, but in 6.1, mNumberBuffers is 2. So that required updating the code where the sample buffers' void*s are saved off to make them available to the render callback.
So, yeah, should never have worked in 6.0 in the first place. I just got lucky.
Finally, because of the inappropriateness of using AUNewTimePitch in an AQTap scenario, I rewrote the demo to use a genuine AUEffect, namely AUReverb2. The GUI has sliders for the Wet/Dry and Gain parameters, so you still get the thrill of messing with audio received from a web radio stream in real-time. So this is what I'm going to point people to for an AQProcessingTap example.
2. AUNewTimePitch distorts audio on the device
Yep, I hear it too. When connected to an AUFilePlayer, from an earlier demo I did, it's pretty obvious.
Someone on the list noted that the effect seems to boost the levels too high. That was was something I noticed too, and the buglet has an audio recording captured from the device with one run through of the loop with the effect on, and a second off. As soon as the effect is off, the levels drop dramatically.
3. AUNewTimePitch should not be used in a constant-time scenario like web radio
…because obviously, if you mess with the rate parameter, you will consume audio faster than it is being supplied and thus run out. Thing is, this code has been shown to work, presumably because it doesn't play with the rate parameter, only pitch. But, since Bill and James advised me to put a buffer up front in order to guard against any problems, that's what I did.
I've updated the code for AQTapDemo-Pitch to include the fixes from AQTapDemo-Reverb but go back to using AUNewTimePitch and a pitch-shifting silder, and also to put a 4KB silent buffer up front so that if the effect ever asks for more samples than were available at the time of the AQTap callback, it can be satisfied. In the updated project, this is available via the #define CALLBACK_PREBUFFER_SIZE. If it's present, I set up a CARingBuffer, seed it with 4KB of silence, and keep separate timestamps for the fetch and store operations. I'm not very happy with this code, but it seems to work. Then again, it also works to just comment-out this #define and use my older approach, which uses only the AudioBufferList provided by the AQTap callback and doesn't do any buffering.
Since I now have the reverb example (#1) for teaching AudioQueueProcessingTap, this web radio pitch-shifter is more of a show-off demo now, and something I can set aside in the future when and if there's an AUPitch on iOS that's a genuine AUEffect, rather than an AUConverter. So it's probably not something anyone wants to copy or put into production… more of a parlor trick.
--Chris
P.S. Also, I was asked by the Monotouch folks (who did a port of the demo code to their environment) to put a really permissive license on this code so people won't have any hangups about using it, so the sample code is all MIT-licensed now. Generally, I mean for anything I put in a book or conference demo to be freely reusable, no attribution, no questions asked. But the Monotouch guys wanted something more formal, and it'd be nice to not be sued if this stuff blows up in someone's face, so MIT license will do.
On Feb 5, 2013, at 8:58 AM, Chris Adamson < email@hidden> wrote: OK, thanks for the info on converter vs. effect. I'll rewrite this demo for future conferences to use something other than AUNewTimePitch.
On Bill's suggestion to prime the queue with empty data, I'll try this when I get home tonight, or just work with DTS when I hear back on the support incident (I have a few questions that probably don't concern the list as a whole).
Should I file a radar to officially ask for AUPitch on iOS?
Thanks again.
--Chris
Sent from my iPad On Feb 4, 2013, at 9:11 PM, William Stewart < email@hidden> wrote: There are a couple of things to note:
For this example, one way you might be able to get it to work is to add the following: (1) Add an intermediary buffer of (we think 4k samples is enough) - in the AURemoteIO, you would cache 4K samples of input (so the first 4k samples you provide for the output is silence) - in the AQTap, you cache the first 4k samples you get from the tap, and push 4k samples of silence into the graph. (2) As long as you do NOT change the rate, we think this number of samples should be sufficient
Looking forward, we should provide an AUPitch (as we have on the desktop) as an effect unit that will - allow you to just change the pitch. - is an effect unit, so behaves in a real-time manner (that is, it only ever asks for the same number of input samples as output samples, so can be used in these sorts of contexts)
Thanks
Bill On Feb 4, 2013, at 2:49 PM, James McCartney < email@hidden> wrote:
On Feb 4, 2013, at 2:28 PM, Chris Adamson < email@hidden> wrote: Well, I'm away from my source for the moment, but I think this begs a question of how you would use an AudioQueueProcessingTap to apply an AUEffect, g
You are not applying an effect, you are applying AUNewTimePitch, which is a format converter. iven that the AUEffect likely uses a different PCM format than what is being received from the tap (and given that the format received by the stream cannot be changed). In my experience, the AUEffects require specific floating-point formats, and I've never been able to set the stream format of an AUEffect and not get -50 (paramErr).
So, given that I cannot change the format I get from the queue, nor can I change the format supplied to the AUEffect, it seems like the only reasonable option is to use an AUConverter.
I should note by the way -- I don't know if you looked at the code -- the actual use of the converter and effect is in an offline AUGraph. In the tap callback, I pull the provided buffer through an AUGraph that looks like this:
[render callback] -> AUConverter -> AUEffect -> AUConverter -> AUGenericOutput
The converters go to and from the AUEffect's default format, and the render callback exists only to provide the buffer that was originally passed to the tap callback. It looks unnecessarily elaborate, but since the format issues prevented me from just calling AudioUnitRender() in place on a single AUEffect, it was the best solution I could get working.
--Chris
Sent from my iPad On Feb 4, 2013, at 4:54 PM, James McCartney < email@hidden> wrote:
You should not put a format converter (kAudioUnitType_FormatConverter) into an AudioQueueProcessingTap. That is only meant for effects. Unlike effects, format converters can pull for unpredictable amounts of audio. The only exception might be an AUConverter that is not doing any sample rate conversion. So it is expected that this will not work.
On Jan 31, 2013, at 8:36 AM, Chris Adamson < email@hidden> wrote: Yeah, I'm looking into the silence issue now. I'm worried that 6.1 may break the Audio Queue Processing Tap (or at least how I'm using it). That demo still works on the 6.0 simulator, but produces silence on 6.1 (simulator and device).
The AUNewTimePitch demos from that CocoaConf Portland blog still work on 6.1.
--Chris
On Jan 31, 2013, at 11:26 AM, Tony Kirke < email@hidden> wrote: I've also had the same issue based on Chris's demo project. In fact I just upgraded to 6.1 and am now getting silence instead of distortion. Were you on 6.0 or 6.1?
On Thu, Jan 31, 2013 at 3:27 AM, Jont Olof Lyttkens <email@hidden> wrote:
Hi!
I am building an app for iOS. which depends to a great extent on time-stretching and I was thrilled to find the NewTimePitch Audio Unit. I am configuring a simple AUGraph:
kAudioUnitSubType_AudioFilePlayer -> kAudioUnitSubType_NewTimePitch -> kAudioUnitSubType_RemoteIO
The parameter "rate" of the NewTimePitch is set from around 0.7 to 1.3. Problem is that the NewTimePitch Unit distorts the audio quite badly. I found this post on coreaudio-api-maillist which relates to the problem:
Unfortunately I don't come to the same conclusion. I don't hear any phase problems, nor any monofication of the signal. We are only talking really bad distortion. I've done testing with all sorts of audio input down to a -24dB FS audio input and the distortion seems to be unrelated to volume.
Finally I found this blog post from CocoaConf by Chris Adamson, where he bundles the demo project from the conference:
I must admit that he pointed out a couple of things that I had missed out, but even after fixing these, I still had the distortion. More so, compiling the Demo Project from CocoaConf also distorted the audio, so I am beginning to wonder if the NewTimePitch AU really is corrupt.
A printout of my CAShow():
AudioUnitGraph 0xBD33001:
Member Nodes: node 1: 'auou' 'rioc' 'appl', instance 0x815e940 O I
node 2: 'aufc' 'nutp' 'appl', instance 0xa85ade0 O I
node 3: 'augn' 'afpl' 'appl', instance 0x8298510 O I Connections:
node 3 bus 0 => node 2 bus 0 [ 2 ch, 44100 Hz, 'lpcm' (0x00000029) 32-bit little-endian float, deinterleaved]
node 2 bus 0 => node 1 bus 0 [ 2 ch, 44100 Hz, 'lpcm' (0x00000029) 32-bit little-endian float, deinterleaved]
CurrentState: mLastUpdateError=0, eventsToProcess=F, isRunning=F
I also have:
- Ensured that the StreamFormat of the AUNewTimePitch is set equally on all other units
- Setup the RemoteIO bus
Is there any magic trick that I need to know about or does anyone have a non-distorting sample of a graph that works with the NewTimePitch Audio Unit?
Best regards
/Jont Olof
_____________________________________
Co-founder and Senior Cocoa Developer centCode AB
Adress: Frödingsgatan 25, Uppsala, Sweden
Zip: SE-75421
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
James McCartney Apple CoreAudio
James McCartney Apple CoreAudio
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
|