• Open Menu Close Menu
  • Apple
  • Shopping Bag
  • Apple
  • Mac
  • iPad
  • iPhone
  • Watch
  • TV
  • Music
  • Support
  • Search apple.com
  • Shopping Bag

Lists

Open Menu Close Menu
  • Terms and Conditions
  • Lists hosted on this site
  • Email the Postmaster
  • Tips for posting to public mailing lists
Re: iPhone hardware assisted AAC > PCM conversion using ExtAudioFileRead
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: iPhone hardware assisted AAC > PCM conversion using ExtAudioFileRead


  • Subject: Re: iPhone hardware assisted AAC > PCM conversion using ExtAudioFileRead
  • From: Adriano Goncalves <email@hidden>
  • Date: Mon, 28 Jan 2013 09:32:22 +0000

No, just a MultichannelMixer and the RemoteI/O.

I thought I could feed the Mixer with the compressed format, so instead of setting the input stream format to LinearPCM I was wondering if I could feed directly the IMA4, but I have to decompress it first, right?

Right now my app have the same memory load as yours, around 130Mb. But in my case I'm putting to much preloaded samples in memory, I don't need to do so. And on the 3GS there are some memory warnings already.


On 27/01/2013, at 06:51, David Blake <email@hidden> wrote:

Adriano, why are you setting the stream format to kAudioFormatAppleIMA4 ? You using a converter audio unit ?

For me I convert all sounds (ima4/aac) to pcm before I feed it into the audio unit chain, so I just use the same ExtAudioFileRead code which will covert accordingly regardless of what format I feed it.

With my experiments a couple of nights ago I found that performance with IMA4 vs AAC was basically about the same in terms of how many sounds are playing before it starts skipping. However to play the potential max amount of sounds on my app on the iphone 4 (about 35 simultaneously) I have found I have had to increase by buffer size by about 4x (to give the iphone 4 enough time to catch up with the converting). Unfortunately this leaves a memory footprint of about 140mb!! So not sure if that is acceptable right now, certainly worked on the iphone 4 without crashing (really ramped up the buffer size to see how much it could take too and it didnt crash and even double what I've set it to). Cant even seem to deploy to a 3gs to test now (think thats to do with arm support), but i know ipad 1  and ipod touch only has 256mb so need to test on those really.

Is 140mb an unacceptable size do you think? Have you heard of any apps taking up that amount?



On 26 January 2013 06:18, Mo DeJong <email@hidden> wrote:
Adriano, you could try this download for an example of using IMA4 in an iOS project:

http://www.modejong.com/iOS/#ex3

Just FYI, I tried IMA4 but I ended up giving up on that format because it can degrade bass sounds in certain cases. I would suggest that you ship AAC files with your app, but then decompress each one when the app first starts up. Then, mmap the specific ones you want to play and mix the different audio tracks either manually or with a mixer component with CoreAudio. You will also find a manual mixer example at the same URL above.

Mo DeJong

On Fri, Jan 25, 2013 at 3:17 AM, Adriano Goncalves <email@hidden> wrote:
That seems to be the way, as stated in the same document:

"
For less memory usage when you need to play multiple sounds simultaneously, use IMA4 (IMA/ADPCM) compression. This reduces file size but entails minimal CPU impact during decompression. As with linear PCM data, package IMA4 data in a CAF file.
"

In the app I'm working now, I'm playing 10 samples at the same time, each with 11 seconds and 3.4Mb, which are stored uncompressed. And I'm preloading another 10 at the same time and so far (with a simple UI, nothing fancy yet) its working in a 3GS.

I will try now with IMA4 compression but I'm having difficulties setting the stream format to kAudioFormatAppleIMA4...



On 25/01/2013, at 07:38, David Blake <email@hidden> wrote:

> Hi Adriano,
>
> Thanks for your reply.. !
>
> I had a go at changing my AVAudioSession. It was previously set to AVAudioSessionCategoryPlayback (which supports hardware encoding/decoding) but I tried changing it to Ambient (which doesn't) to see if there was any noticeable performance difference. Results did not seem to be any slower than having it set to Playback however. Which leads me to believe that regardless of the AVAudioSession, ExtAudioFile still uses software codec for decoding. All references to hardware assistance with ExtAudioFile that I have come across indeed all point to encoding only. I tried setting hardware codec manually anyway incase they were using encoding and decoding synonymously but it didnt seem to affect performance (see below for code).
>
>     UInt32 codec = kAppleHardwareAudioCodecManufacturer;
>
>     UInt32 size = sizeof(codec);
>
>     result = ExtAudioFileSetProperty(audioFileObject,
>
>                                      kExtAudioFileProperty_CodecManufacturer,
>
>                                      size,
>
>                                      &codec);
>
>     if(result) printf("ExtAudioFileSetProperty %ld \n", result);
>
>
> Only way to get access to hardware decoding I have found is to use AudioQueues, which stop me if I'm wrong you cant use if you are using Audio Units (I presume as queues are just a higher level wrapper for units).
>
> Thinking I might abandon trying to use AAC files and maybe switch to a lighter compression such as IMA4, wont be as small but wont be as huge as PCM either :) hopefully it'll be less cpu heavy to decode.
>
>
>
>
> On 23 January 2013 20:32, Adriano Goncalves <email@hidden> wrote:
> According to this document: http://developer.apple.com/library/ios/documentation/audiovideo/conceptual/multimediapg/multimediaprogrammingguide.pdf
>
> "
> When using hardware-assisted decoding, the device can play only a single instance of one of the supported formats at a time. For example, if you are playing a stereo MP3 sound using the hardware codec, a second simultaneous MP3 sound will use software decoding. Similarly, you cannot simultaneously play an AAC and an ALAC sound using hardware. If the iPod application is playing an AAC or MP3 sound in the background, it has claimed the hardware codec; your application then plays AAC, ALAC, and MP3 audio using software decoding.
> "
>
> In the same document there's a table with all the formats that support hardware decoding.
>
> Also, I believe you should set your audio session category:
>
> http://developer.apple.com/library/ios/#documentation/Audio/Conceptual/AudioSessionProgrammingGuide/Configuration/Configuration.html
>
> Hope this may help you finding the solution for your question!
>
>
> Adriano
>
>
>
> On 23/01/2013, at 00:46, David Blake <email@hidden> wrote:
>
> > In my iphone app I have a large number of PCM sound files that can be played at any time. I load the first few frames of each sound file into memory and then as they are played I load the next frames into a buffer that is read by a render callback in remote io.
> >
> > Problem is these files take up a lot of space and ideally I would like to reduce the size of my app by storing these in a compressed AAC format and decoding this on the fly as needed. Right now I am using ExtAudioFileSeek and ExtAudioFileRead in a background thread to decode the next buffer to be read. When doing this with AAC files right now it pushes the iPhone 4 to its absolute limits when playing the max number of potential sounds at once. I know the iPhone has a hardware decoder for the AAC format - how can I know if this is being used, or is it even possible to use this when doing offline decoding/converting via ExtAudioFileRead ?
> >
> > If it is using software decoding by default, my hope is that setting ExtAudioFileRead to use the hardware decode might alleviate some of the cpu stress on the iPhone 4.
> >
> > I have read about the kExtAudioFileProperty_CodecManufacturer / kAppleHardwareAudioCodecManufacturer property that be set on ExtAudioFile but have yet to find example code of applying this that actually works, or if it only applies to encoding and not decoding. Indeed I came across this old thread http://web.archiveorange.com/archive/v/q7bubfjRLCAE0t1Q38V2 which suggests its encoding only for ExtAudioFileRead. If this is the case are there alternative methods to doing offline hardware assisted decoding of AAC files (that are compatible with my remote io setup) ? If I am to believe that thread than audio queues are the only way to get access to the hardware decoder.
> >
> > Any advice/help would be greatly appreciated.
> >
> > Cheers,
> > David
> >
> >
> > _______________________________________________
> > Do not post admin requests to the list. They will be ignored.
> > Coreaudio-api mailing list      (email@hidden)
> > Help/Unsubscribe/Update your Subscription:
> >
> > This email sent to email@hidden
>
>


 _______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:

This email sent to email@hidden



 _______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:

This email sent to email@hidden

  • Follow-Ups:
    • Re: iPhone hardware assisted AAC > PCM conversion using ExtAudioFileRead
      • From: David Blake <email@hidden>
References: 
 >iPhone hardware assisted AAC > PCM conversion using ExtAudioFileRead (From: David Blake <email@hidden>)
 >Re: iPhone hardware assisted AAC > PCM conversion using ExtAudioFileRead (From: Adriano Goncalves <email@hidden>)
 >Re: iPhone hardware assisted AAC > PCM conversion using ExtAudioFileRead (From: David Blake <email@hidden>)
 >Re: iPhone hardware assisted AAC > PCM conversion using ExtAudioFileRead (From: Adriano Goncalves <email@hidden>)
 >Re: iPhone hardware assisted AAC > PCM conversion using ExtAudioFileRead (From: Mo DeJong <email@hidden>)
 >Re: iPhone hardware assisted AAC > PCM conversion using ExtAudioFileRead (From: David Blake <email@hidden>)

  • Prev by Date: Re: How Can I Determine When a ScheduledAudioFileRegion Has Finished Playing?
  • Next by Date: User mode driver (HAL plugin) vs Kernel mode
  • Previous by thread: Re: iPhone hardware assisted AAC > PCM conversion using ExtAudioFileRead
  • Next by thread: Re: iPhone hardware assisted AAC > PCM conversion using ExtAudioFileRead
  • Index(es):
    • Date
    • Thread