Re: Convolution Audio Unit how to?
Re: Convolution Audio Unit how to?
- Subject: Re: Convolution Audio Unit how to?
- From: Brian Willoughby <email@hidden>
- Date: Fri, 25 Nov 2011 13:21:25 -0800
On Nov 23, 2011, at 03:26, Mark Heath wrote:
I have only described my initial problem. I haven't begun to
describe my buffering, I wasn't sure if it was implemented
correctly. I wondered how I determine if my Filter was Process()
ing the left or right channel. This appears to be a moot point as
the host application (or parent class or whatever) starts an
instance per channel.
It's not the host application that handles this, but the parent
class. The source to the parent class is part of your project, so it
is entirely under control of the AudioUnit source code, even if you
normally do not see or change that code.
Also I didn't know how to correctly handle the samples in the first
call to Process() as I need to read ahead windowsize/2 samples. So
I would only be able to output inFramesToProcess minus windowsize/2.
You must always output inFramesToProcess. The first call is not
really handled any differently than the rest. Technically, the Reset
() call should fill your private buffers with silence and set the
counters to zero or some constant that will allow Process() to
function correctly.
eg if the partition size was 512 samples and my window size was 16
samples. then I could only output 504.
You seem to still be assuming that your algorithm operates directly
on the incoming data. It does not. You must copy the incoming data
to a FIFO where it will then be processed by your algorithm. The
FIFO consumes from the input buffer and feeds your convolution. Your
core convolution code consumes from the input FIFO and feeds the
output FIFO (you might be able to combine the two FIFO memory
elements if you're clever). Finally, your AU consumes from the
output FIFO and feeds the AU output buffer. Between the AU I/O and
the FIFO(s), you're dealing with 512 samples in your example.
Between the FIFO(s) and your convolution code, you're dealing with 16
samples at a time. You must implement counters to track all of this.
It appears that I simply zero pad the firs 8 samples of the output
partition and set a latency (of 8 / samples per second). correct?
No. You're missing the fact that you need to keep around samples
from preceding calls to Render() so that your window is always
contiguous.
Also, why did you start with an example of a 16-sample window, but
now are talking about 8 samples of padding and latency?
I was under the false assumption that I must process input sample 1
into output sample 1's position but wondered what I did towards the
end of the input buffer. I guess that this assumption is only a
requirement for non realtime filters, that I don't output anything
until I have read enough input. This might be a flawed when
applying it to Audio Units (or other realtime audio frameworks).
If you report a latency of 0.0, then you must process input sample 1
into output sample 1's position. However, since convolution
necessarily involves latency, then you will never be able to have 1-
to-1 sample alignment. Instead, your AU reports its latency, and
that tells the AU host that the sample positions will not be
aligned. Then, your job is to implement code that processes input
sample 1 into output sample 16's position, assuming a latency of 16
samples. It has nothing to do with real-time or not.
Is this true, that for filters which require look ahead the very
start of the output must be padded with 0s?
Yes. There are many ways to implement the details, but they all end
up with silence at the beginning for the duration of the latency.
The Reset() call is the only clue that the AU has as to when the time
line "begins," so your algorithm and state variables must be adjusted
properly in the Reset() call.
If I report 0 as tail time does this mean that I will lose
windowsize/2 samples from the end?
My understanding is that the AU host will combine latency and tail
time. If you report your latency as windowsize, then the AU host
will continue to pull from your AU after the end for windowsize
samples. If the tail time is 0, then it stops there, but you haven't
lost anything.
This is interesting as the window size is a user settable
parameter. According to the example I must be able to handle the
user changing these parameters while the filter is running. Does
this mean that I cannot change the latency? Or that I must write
my filter in such a way that the latency is fixed regardless of the
parameters?
You cannot change the latency while the AU host is running, because
it will not know that anything changed. You can notify the AU of a
change to the latency parameter, but it's doubtful that any host will
expect latency to change. Instead, there is an "initialized" state,
before which the AU host is supposed to give your AU all the
information it needs, such as sample rate and buffer size, so that
you can report the proper latency.
Note that latency has nothing to do with the user settable CoreAudio
buffer size. If your actual convolution window size is truly user
settable, then you'll need to send a notification whenever you change
the latency, and you'll probably also need some mechanism to make
sure that this only happens when the transport is not running.
That's actually a difficult challenge considering that you do not
have the basics working yet.
I've made a buffering implementation that assumes that the
inFramesToProcess is larger than my window size. There is
possibly a case where this is not true, which I have not written for.
Bad assumption. Sample-accurate parameter rendering is a feature
that an AU host might implement without notification to your AU other
than the parameters to the Render() or Process() call. Your code
should be prepared to render a single sample, if necessary, which
highlights the need for a separate FIFO beyond the normal buffers
that are provided as parameters.
How does the framework handle no output samples? Zero padding?
The frameworks does not handle "no output samples" because it does
not allow this. You must always produce the requested number of
samples. The understanding is that a plugin with latency will supply
silence whenever the transport restarts, until the latency time has
transpired.
I assumed that if I was given 512 samples to process that I must
output 512 samples.
Correct. You have the basic idea except for the time shift due to
latency.
Which is why I was able to produce a working concept that shows
that the theory is correct, but because I don't process the edges
of the partitions, it generated distortion.
Again, you absolutely need the FIFO so that samples do not get lost.
When, e.g., 512 new samples come in, you cannot process the final 15
samples if your window is 16 samples long. So, at the very least,
you need to save those 15 samples in a FIFO so that they're still
around again when you get the next 512 new samples. On that
subsequent call, you'll actually still be processing 15 or 16 samples
from the previous call, along with all of the new samples.
Think of it as a chain: Input buffer to input FIFO, input FIFO to
convolution, convolution to output FIFO, output FIFO to output
buffer. The FIFO sticks around between calls to Render/Process so
that you do not get distortion.
There are a number of ways to handle this, but you basically need
a FIFO that is large enough to hold everything that your
convolution calculations need, plus you must have some sort of
counter to keep track of how much input and output data is waiting
in the input and output working buffer (FIFO).
I can see that this would be needed if the window was larger than
the partition size. Maybe I need to take another step back and
implement this as a generic case rather than my current
implementation that assumes my window will be smaller than the
partition size.
The window is always larger, because there is no guarantee that you
won't be called upon to produce just 1 sample.
In such a case, your input FIFO would have 1 sample added, at which
point your convolution may or may not have enough samples to run a
process that would add a window of samples to your output FIFO. Your
output FIFO, therefore, has better have at least 1 sample waiting
even before the convolution runs, otherwise your output will glitch.
Brian Willoughby
Sound Consulting
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden