Re: Decimal precision in Photoshop's info palette
Re: Decimal precision in Photoshop's info palette
- Subject: Re: Decimal precision in Photoshop's info palette
- From: Graeme Gill <email@hidden>
- Date: Sun, 21 May 2006 22:33:06 +1000
Dennis W. Manasco wrote:
At 11:09 PM +1000 5/20/06, Graeme Gill wrote:
The ratio is actually 257. (255 * 257 = 65535)
No. Remember that 0 is a valid and necessary input value. Thus 0..255
has 256 discrete values and 0..65535 has 65536 discrete values.
Sorry, you must live in a different universe to me. In my
universe 255 * 256 = 65280.
Thus 256 * 256 = 256 * 2^8 = 65536.
Which is irrelevant, since 8 bits can't represent 256, and
16 bits can't represent 65536.
8 bits can represent 0 to 255, and 16 bits can represent 0 to 65535.
The CPU's multiplier works just like normal arithmetic in the real world.
Binary 0xff (the maximum 8 bit unsigned value) is the number 255 to the multiplier.
Binary 0xffff (the maximum 16 bit unsigned value) is the number 65535 to
the multiplier. The ratio is 257. That's how multiplication works.
(You've let the fact that there are 256 8 bit values, and 65536 16 bit values
lead you astray.)
My point was that 0..100 does not map explicitly onto unsigned integer,
while 0..255 does.
I'm not sure what you mean by "explicitly"
If you enter a number in the range of 0..100 for an input that will be
interpreted as unsigned integer you have _introduced_an_error_ in the
program's evaluation of your intention _from_the_very_beginning_.
Not at all. You have specified what you want, the program interprets
it in light of what's possible. If I specify 50.0%, and the underlying
system is 8 bits, I get 128, and error of about 0.2% (round to nearest (0.5 * 255)).
If the underlying system is 16 bits, I get 32768, an error of about 0.00076%.
Both of these are the worst case quantisation error.
If I specify 50.0% and the underlying encoding is Adobe's 16 bit
as used in photoshop (0 to 32768), I get 16384, and error of 0.
If I want a specific device value, I can relatively easily
get it (although a direct interface is better in this special situation).
For instance, if the underlying system is 8 bits, and I want 117, I can
enter 45.9%.
An error which the program will compound with every step of its
manipulation and further evaluation.
It depends a lot on what the users expectation is. Most users would
really not like to have to know the minutiae of the underlying system,
so if they specify 50%, they'd like the system to do it's best
to achieve that, and for many purposes, an accuracy if 0.2% or better
is sufficient (after all 50% has only 2 significant figure, so
the user might have meant any value between 49.5 and 50.5%).
The quantization error does not get "compounded". It isn't marked as
special in any way, once the number is in the underlying
representation, it is treated the same as any other number, and
exactly the same errors befall it as a number entered by a different
means.
After all, 30% cyan is 76.5 on a scale of 0..255. If the program is
representing your cyan in 8-bits it's _never_ going to get that right.
It will get it right within 0.2%, the very best that can be achieved
with 8 bits. If you want better than that, you have to use a higher
precision underlying representation, and then you can simply specify
30%, rather than having to calculate some other number.
I am not sure what you mean by a "real value."
As in, not an integer.
Yes, 40% is a lot easier to think about and use than 102, but it's only
blind luck that 40% maps directly onto the range of 0..255.
So what if it maps exactly to a device value with zero error ? If the
user wants 30% instead, then there is nothing they can do if they are stuck
with that underlying representation, and 30% doesn't map exactly. Being able
to enter a device value directly doesn't change the situation.
They can enter 76 or 77, neither of which is 30%. What difference
does it make ?
What if you wanted 10%? Oops. That's 25.5 on a 0..255 scale. Your result
will always be off your desired value by 1.96%. I don't know of any way
to ascertain the direction of the error.
By convention you round up. Again, what difference does it make if
you enter a device value ? You can use 25 or 26, neither of which is
your 10%.
If you mean that I should have to enter 49.8039215686275 when I want a
value of 127 on an integer scale of 0..255 then I respectfully decline.
You can enter 49.8, or 49.7, but why do you want 127 ? What's special about it ?
Will you notice the difference between 127 and 128 in most situations ?
If it is a situation in which you will notice the difference, why are you
using 8 bits ?
Let's argue for floating-point representation of color at every stage in
PhotoShop (including dialog entry), except for output to other formats
and printing.
It comes down to whether you are prepared to pay the price in terms
of performance and memory usage (and possibly disk space.)
Consider that many people stick to 8 bits because it's "good enough",
while 32 bit floats would take 4 times the memory, and would probably
be noticeably slower.
Graeme Gill.
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Colorsync-users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden