Re: PThread - Core Audio Callback being missed on high load
Re: PThread - Core Audio Callback being missed on high load
- Subject: Re: PThread - Core Audio Callback being missed on high load
- From: Jeff Moore <email@hidden>
- Date: Fri, 18 Nov 2005 12:45:34 -0800
It sounds very much like your IOProc is taking too long on occasion,
despite your optimizations. You don't mention how you are
implementing your buffering with respect to synchronizing it with the
other threads. Are there any blocking calls in your IOProc, like
locking a mutex or anything like that?
The best tool to help you out is HALLab's IO cycle telemetry viewer.
You can use that to tap into what your app is doing from the HAL's
point of view. It's easy to see the overload and the basic reason why
it happened from the telemetry data. Further, should it turn out to
be something other than your IOProc, like a scheduling latency or pre-
emption or what ever, HALLab can also generate kernel traces for the
overload so you can see what was going on from the kernel's point of
view for the overload.
Also, you seem to be making a bad assumption about processor usage
with respect to real time programming. Monitoring usage with top or
Activity Monitor or Shark just shows you average load. Average load
isn't a particularly useful stat with regard to code that has to meet
a deadline. The important figure is peak load. Everything you said so
far indicates that you still most likely still have something in your
code that is causing your IOProc to take too long. This quite often
is too small to be even a blip in an average load measurement, but
still causes an overload.
On Nov 18, 2005, at 6:18 AM, Mark Gilbert wrote:
Folks.
Our application is a 128 track audio recorder. We use CoreAudio
(with device aggregation) to deliver a single 128 channel audio
stream into our app via the normal CoreAudio callback installed
with 'AudioDeviceAddIOProc'.
On a dual G5/2.7 our app copes perfectly with a 64 channel load.
However, when we move up to 128 channels things start to fall apart
in terms of performance.
The app has 3 main processing parts, the IOProc itself which takes
in 128 channels, repackages the data slightly and meters the
inputs. This IOProc has been fairly well optimized now and is
pretty light. The IOProc seems to run in a separate thread created
by CoreAudio. The audio is fed into a very large Float32 circular
buffer (up to 30 seconds worth).
We then have a function which is called from our main thread as an
idle Proc which (once per second) reads from the circular buffer,
converts the data to 24 bit integer and writes the data into 128
files. We have tried telling OSX not to cache its writes, although
we have not tried async writing yet.
The total system load is less than half what is available. We are
not running out of CPU Performance, but we are running out of time
(something is holding us up randomly).
Now. The problem we see is that our IOProc should be called about
every 10 mSecs with new incoming audio packets. What happens is
that one of the IOProc calls is skipped, and we get called after 20
mSecs instead, with the consequential loss of a packet of audio.
Prior to our optimizations of the IOProc, we would see this happen
regularly, even when we were NOT writing to disk. Now that its
optimized, basic playthru of the data is OK. However, when we
write to disk, we now see the IOProc skipping again sometimes.
Note that the IOProc is a separate PThread, created by CoreAudio,
whilst the disk write is executing inside our idle timer from the
main thread. There is so much circular buffer, there is no risk of
an underrun.
*** The entire problem is caused because when this problem happens,
our IOProc is not called and we miss the opportunity to feed
incoming data into the buffer. ******
As I say, CPU usage is comfortable, so we are not simply running
out of CPU time. It seems that something is blocking the CoreAudio
mechanism which would call our IOProc. I have optimized everything
I can , but we are stuck with this problem.
The Audio hardware we are using is 2 x RME Audio MADI PCI Cards in
an OSX Aggregate device configuration. MOTU Digital Performer
(and Logic Audio) both use this same device successfully on the
same machine so we know that its possible.
Apple's ComplexPlayThru example glitches like crazy (like mine did
before I optimized) and the SimplePlayThru example does not pass
any audio with this device.
My questions are :
- Is there some way that we can raise the priority of the IOProc so
that it gets called even when other things are working hard ? In
this scenario, the IOProc is the most important thing. Of course,
I dont know whether the problem actually lies downstream in the
device driver missing the chance rather than my IOProc being missed.
- Any other suggestions on how to keep my IOProc being called when
the going gets tough. As mentioned the total system load is only
about 50-60%. I have used shark to profile what is happening,
with nothing obviously holding us up. It seems to be some kind of
schedule missing thing.
Thanks for any real time wisdom you can share, particularly anyone
who can speculate on where the hold up may be happening (kernel
side ?)
--
Jeff Moore
Core Audio
Apple
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden