Re: How to capture the audio of an output device in OSX
Re: How to capture the audio of an output device in OSX
- Subject: Re: How to capture the audio of an output device in OSX
- From: Zack Morris <email@hidden>
- Date: Tue, 09 Sep 2014 08:05:23 -0600
On Sep 9, 2014, at 6:57 AM, Frederic De Jaeger <email@hidden> wrote:
> Hi
>
> Hear (http://www.prosofteng.com/products/hear.php) does exactly this. This is an horrible hack. They use Application Enhancer (http://cocoadev.com/ApplicationEnhancer) to monkey patch some CoreAudio APIs.
>
> By the way, Apple, it would be very nice to have an official hook in the CoreAudio pipeline to intercept the audio stream (without creating a dummy driver or monkey patching like Hear). There are a lot of (legal) uses for that.
>
> Fred
I just want to chime in and say that one of the use cases for this is echo cancellation for VoIP. I got pretty far with Speex in a shareware game several years back but without having an easy way to capture all of the sound output (including game sounds) there was really no way to adequately cancel the background noise for multiplayer. I really needed to be able to capture the output of OpenAL as a single stream, run it through Speex’s cancellation, and then send that to the speakers. I finally got so frustrated that I gave up on it. One of a dozen reasons why I never finished the game, but it still cost me days of consternation.
P.S. If Apple is concerned about the security implications of applications capturing the output of other applications, then there are other approaches that would work. They could either use the security APIs to ask the user for permission to use the output stream (kind of a poor user experience IMHO), or provide a pre-cancelled microphone stream. A pre-cancelled stream would actually be simple to implement and there are various algorithms that do a good job. The one in Speex at that time simply subtracted the output from the input, but required precise synchronization of both streams through the audio driver, which may not have been as easy to access on the Mac as on the PC. It might be better to do it in the frequency domain or find the phase there, but anyway I’m sure they could be very good and perhaps the user could be given a choice of algorithms based on CPU usage on both Mac OS and iOS. Apple will probably want to stay on top of this for things like VR because of the precise timing needed to prevent vertigo, for example with the Oculus Rift, Samsung Gear VR or Google Cardboard. With better positional audio (like doppler in OpenAL), echo cancellation could come basically for free. Otherwise I wonder if developers will have to use an API above the OS, for example Unity or Google Chrome. If Apple already provides an echo-cancelled input or are working on some of this next-gen stuff, perhaps they could shed some light here.
Zack Morris
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden