Re: Preroll semantics & clarification of kAudioUnitProperty_OfflineRender
Re: Preroll semantics & clarification of kAudioUnitProperty_OfflineRender
- Subject: Re: Preroll semantics & clarification of kAudioUnitProperty_OfflineRender
- From: Heinrich Fink <email@hidden>
- Date: Wed, 23 Nov 2011 11:47:55 +0100
Hi,
> On Nov 23, 2011, at 06:10 , Brian Willoughby wrote:
> Since the design goal of the OfflineRender property is to increase the render quality, I see no reason that you would need your preroll audio to be rendered to a higher quality than the realtime audio. Basically, you should run everything at Online Render quality and just call that good enough.
>
> In other words, your goal all along has been to achieve higher rendering speed during preroll, not higher rendering quality. Since OfflineRender does not increase rendering speed (but more likely slows it down in most cases where it makes any difference at all), then it seems like your application would want to specifically avoid OfflineRender (unless I missed something).
I am curious how many audio units actually support the OfflineRender property at all (in the realm of audio effects). As you said, preroll does not require better quality than realtime rendering, but the exact same audio quality as online rendering with faster rendering speeds (unlike bouncing). If a particular audio graph setup would not be able to satisfy the 2 sec. preroll phase in a reasonable amount of time, I would then handle this similarly to a general audio overload. Most of our users in broadcasting are unlikely to use a heavy effects chain anyway (unlike users of an audio sequencer).
So just ignoring the OfflineRender property and to hope for the best seems to a possible but potentially unsafe approach. When implementing it this way, we would have to test the possible permutations of the audio graph beforehand and only expose a particular set of audio unit effects to our users which we have confirmed to work without glitches during and after preroll phase. This of course is both limiting the user’s experience and also increasing the workload for testing and maintenance on our side.
> On Nov 23, 2011, at 10:19 , Stefan Gretscher wrote:
> In a music production software for recording studios, you can get away with realtime render and occasional dropped buffers during preroll - the affected tracks will most likely just start playing a little later than the other tracks. However when working on broadcast software, dropping buffers when starting audio is no viable option, and in this use case I don't see any way around switching to offline rendering for the pre-roll if you want to it faster than realtime.
This is a good point. In my understanding, the whole idea of executing a preroll phase first, instead of directly using a realtime context is to have a safety buffer available that compensates for hiccups in the processing chain. This should avoid dropped frames which might cause artifacts or even worse: out-of-sync playback. In other words, the point of having a preroll buffer in first place, is that you DON’T have a system that is capable to operate in real time, unlike AudioUnits which are mostly fully functional in a real-time context.
Now, if using preroll scenario for my software design introduces less determined behavior and even more potential glitches than using realtime only, it makes me think if it weren’t better to just avoid the preroll phase at all. I think the root of this problem is that I am trying to connect a system that is primarily designed to be used in a realtime context (audio units) with an adapter (the preroll buffer) that was primarily designed to be used by non-realtime systems in order to enable realtime playout. From this perspective it seems unnecessarily complicated and is probably the reason why my use case just doesn’t really fit with AudioUnits (or at least isn’t very well supported).
While native SDKs of most video broadcasting cards that we use require a preroll phase, these cards usually provide a CoreAudio hardware device as well (at least for AJA and BlackMagic cards). In order to ensure proper sync between video frames and audio we would have preferred to use the SDKs directly - preroll already couples audio chunks with video frames. But I guess if they implemented their CoreAudio drivers properly, i.e. reported hardware latencies are correct, we should be able to separate audio playout from video and yet ensure correct timing. After all, the CoreAudio hardware should provide us with enough information to determine accurately when audio is going to on the wire.
So I have a feeling that spending more effort on implementing accurate realtime scheduling for an audio path that is decoupled from video and its preroll phase - i.e. using the CoreAudio device directly which wouldn't require any faster-than-realtime rendering with unclear behavior - might be the preferable path to follow. This is my first project in broadcasting, so I’m not sure if this is a reasonable change of path, or if I just started digging my own grave :) Any comments on this alternative solution?
With the decoupled approach, there would of course still be the option of going completely offline and to feed the preroll buffers only. Realtime monitoring (e.g. on a separate aux bus) would then be much harder of course. Also, any additional time required by audio units to properly render would then have be covered by the preroll buffer.
best regards,
Heinrich Fink _______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden