Re: 64bit processing - Why?
Re: 64bit processing - Why?
- Subject: Re: 64bit processing - Why?
- From: Ethan Funk <email@hidden>
- Date: Thu, 30 Jul 2009 21:15:12 -0700
After giving the dithering process a bit of thought, I realize the
error of my thinking... you are intentionally introducing shaped
additive noise at the LSB level every time. So after doing a truncate
and dither 16 times, you have pushed your quantization noise floor up
to the 20 bit precision level.
So I asked my self why not just process with doubles and truncate the
mantissa back to 24 bits before going out with out a dither... The
use of floating point format will keep all your mantissa with in 1 bit
(6 dB) of 24 bits regardless of the level by virtue of the power of 2
exponent, so even a very small signal going through, for example an
AGC processor, will have close to full 24 mantissa bits going in, and
then after the AGC normalizing, those same 24 bits going out, with
only the exponent adjusted... IF the AGC gain is in 6 dB steps.
Intermediate gains will of course effect the mantissa bits, but that
last 6 dB is lost in quantization noise any way. No problem.
Then I though about a processing unit that would square the signal.
Now I see a problem with my truncate approach:
Given S as the mantissa of the input signal (always large - with in 1
bit of full scale - in a floating point format) and N as the
quantization noise of that mantissa,
(S+N)^2 = S^2 +N^2 + 2*S*N
The N^2 is not a problem as N very small and squaring it will make
really really small. The 2*S*N however is a big problem... we just
doubled the noise. The use of doubles when performing that math does
us no good. The precision of the source is the key. One bit of
precession from the source will be lost every time you square it even
if the squaring is done with doubles. So, I'm sold! There is a
justification for double precession between processing units. But I
do want to point out that extra precession is useless if the actual
signal it represents contains noise above the 24th mantissa bit. It
will be that noise level that will dominate the N in the above example
- a squaring function like this will double the analog source noise
too. It would be a noisy plugin by virtue of the math. You better
have a good ADC.
Thanks everyone for helping me work thought this!
Ethan...
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden