Re: Question regarding recording and accounting for drift
Re: Question regarding recording and accounting for drift
- Subject: Re: Question regarding recording and accounting for drift
- From: Neil Clayton <email@hidden>
- Date: Tue, 13 Jan 2009 15:03:26 +1300
- Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys
Hi Brad,
Thanks for taking a look.
I understand what you mean by dropping frames (that'd it be easier).
If however I didn't want to do that is using a varispeed a decent
approach? Is setting it based on the mRateScalars of the two devices
in question accurate over the long term (hours)? What kind of
techniques do CA/QT use to synchronize multiple audio sources (where
dropping frames isn't an option)?
Isn't it reasonable to assume that the output device and the
AudioConverter (the two "output" devices in my diagram) will always be
slightly out of sync? The AudioConverter would be exactly 44.1 (for
example) and the output device would be very very very close to that,
but likely not exact over long periods of time, and so wouldn't kind
of synchronization/resampling be necessary?
Is it valid to store the sampleTime and hostTime of the first sample,
then measure total drift as:
Float64 elapsedSampleTime = inTimeStamp->mSampleTime - firstSampleTime;
UInt64 elapsedHostTime = inTimeStamp->mHostTime - firstHostTime;
QTTime elapsedSampleQTTime = QTMakeTime((long long)elapsedSampleTime,
outputFormat.mSampleRate);
QTTime elapsedHostQTTime = QTMakeTime((long long)elapsedHostTime,
(long)AudioGetHostClockFrequency());
QTTime diff = QTMakeTimeScaled(elapsedHostQTTime,
outputFormat.mSampleRate);
diff.timeValue -= elapsedSampleQTTime.timeValue;
In words, the difference is equal to the distance between the
sampleTime and hostTime (where hostTime is scaled to sample time).
The rational being that a) hostTime is just that. host time. b)
sampleTime is relative to the real device and reflects time as it
occurs according to that device (and therefore is subject to drift,
since it's not mastered to the host clock).
Neil
p.s. I know this could be seen as going about things "the hard way". I
don't mind that, I learn more this way.
On 13/01/2009, at 8:17 AM, Brad Ford wrote:
On Jan 11, 2009, at 8:06 PM, Neil Clayton wrote:
Hello all, and a happy new year!
I've written a bit code that can record from some device, to a QT
Movie. It's using a couple of AU's, a home grown circular buffer
and Audio Converter. It'll also let you playthru (much like
CAPlayThough) if you're recording via Soundflower (or
AudioReflectorDriver).
I'm seeing some drift. Probably not surprising since I've not
accounted for it on the recording side.
I'd like to ask some more experienced developers if my current
reasoning seems valid:
Diagram here:
http://dl.getdropbox.com/u/421935/Audio Recorder Design.pdf
This example shows recording from a 10ch device, playing through to
a 2ch output (Speakers) and recording the 10ch discretely into QT.
The dashed lines show my proposed change below.
The input / output flow is for plauthru.
The input / recorder flow is what I'm concentrating on.
Assuming that the circular buffer places data in "time" according
to the timestamps received, if the input source were running
slightly faster than the Audio Converter itself, would it be
reasonable to assume that over time the audio would get ahead of
the video (not shown)?
So would it therefore be reasonable to add another varispeed before
the AC unit, as a way of controlling this? Would that be enough?
The input/output rate of the varispeed and input rate of the AC
would all match the nominal output rate of the Input HAL. I'd then
account for drift using a method similar to what I've seen in
CAPlayThrough (primarily because it's all I've got to go on).
Am I barking up the right tree?
So...you're recording from 1) an audio input device and 2) a video
input device to file and also performing a real-time preview to 3)
an audio output device, and 4) the screen. That means you've got 4
different devices with potentially 4 different clocks involved
(unless 1) and 2) are coming from a muxed device). I'd recommend
syncing 2) to 1), since it's easier to duplicate or drop (or change
the timing of) video frames than to resample audio. That will get 1
and 2 onto the same timeline. You could drive a timebase off of
1)'s clock -- that will be your master. For preview, you're going
to probably have to use that master timebase to correct for timing
differences on the output devices -- in other words you're going to
have to resample the audio for audio preview, and you're going to
have to correct the video (duplicate or drop, or change the frame
durations) to keep everything in sync.
Hard problems. This is exactly the kind of thing QTKit capture does
well behind the covers. Are those API's not sufficient for you?
-Brad Ford
QuickTime Engineering
Neil Clayton
email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
@apple.com
This email sent to email@hidden
Neil Clayton
email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden