Re: Media Testing for maclife.de
Re: Media Testing for maclife.de
- Subject: Re: Media Testing for maclife.de
- From: Uli Zappe <email@hidden>
- Date: Wed, 10 Sep 2008 06:10:38 +0200
Am 10.09.2008 um 02:36 schrieb Chris Cox:
They can (meaning they provide better results than e.g. Adobe's
dual approach default profiles in *every single* lighting condition
from bright sunlight to a halogen lamp in a dark room) - that was
one of the more interesting results of my test, and I reported it
here. Did you actually read anything of my review summary?
A single ICC profile cannot account for multiple lighting conditions
- that is one of the big, well known, well understood limitations of
ICC profiles.
And wrong, as my measurements revealed (in the meaning I explained
above).
Of course, this result is remarkable, but the numbers are clear: my
best ICC profile was better in *any single* lighting condition than
e.g. Adobe's default profiles. See my upcoming report, where I will
publish the numbers.
Another thing is to explain these surprising numbers.
Let me point out that Capture One PRO (which I used for my ICC
profiles) performs the white balance adjustment *before* the ICC
profile assignment. It's clear from this processing sequence that the
ICC profile has no way to compensate for peculiarities of the spectral
response of the camera sensor, since after the white balance
adjustment, these have "moved".
So what is it that the ICC profile improves? The only explanation I
have that it's something like (for lack of a better word) a general
"color character" of a camera, e.g. the prevalence of a certain color,
no matter under which lighting.
It is true that my ICC profile, built with a target shot at 6500 K,
provides better results for an image shot in sunlight than an image
shot in halogen lighting. *But* even for an image shot in halogen
lighting, the results for an image with the ICC profile are still
better than those for an image with the best Adobe profile I could get.
So, you might say that the ICC profile doesn't really account for
multiple lighting conditions, it's just that the ICC profile is so
much better all in all than an Adobe profile (for whatever reason)
that even when it provides suboptimal results for lighting conditions
it was not built for, it's still better than the Adobe profile.
But even this isn't the whole story. To explore this further, I built
two profiles with the DNG Profile Editor and a ColorChecker 24. I just
used the target images with the DNG Profile Editor and made no
additional adjustments. I built one single-matrix profile, using only
one target image shot at 6500 K, and one dual-matrix profile, using
two target images shot at 6500 K and 2850 K. I then tested the two
profiles for images shot under various lighting conditions.
Obviously, you would assume that the dual-matrix profile fared better
on average; however, surprisingly, the opposite was the case (though
not by much).
So I can only conclude that while the idea of dual-matrix profiles
sounds nice in theory, it does not really work in practice, at least
not in its current implementation.
How did you account for the different light sources using a single
profile?
Apart from the white balancing before the profile assignment, not at
all.
Or were you using an output (artistic intent) profile
What's an "artistic intent"? I've never heard that expression.
not a camera calibration profile?
I used an ICC camera profile built by ProfileMaker.
If this is the case, then why do the best custom ICC profiles
deliver better results metrologically than any of Adobe's profiles?
(In *various* lighting scenarios!)
I don't know how they could, since the ICC profiles cannot handle
multiple lighting conditions by themselves.
See above.
Most likely you are not telling us something important about your
test procedure.
I don't know what this could possibly be, but wait for my upcoming
report where I will try to describe my procedure in exact detail
??? Adobe profiles can also only be used with the Adobe RAW
converter.
No, they can be used with any RAW converter that implements the
(public) DNG 1.2 specification.
And what are the RAW converters apart from Adobe that implement the
DNG 1.2 specification?
But you did no testing to see if DNG profiles could exceed the
quality/accuracy of the default rendering...
You mean by individually tweaking them?
If so, I already wrote why this approach is completely unfeasible
in a
comparative review of this scope.
In which case you cannot claim the conclusions you stated.
You did not test the DNG profiles to see if they could exceed the
quality.
This is an important point to get clear.
You suggest I should try and *manually* improve on the quality of the
DNG profiles Adobe delivers.
This is an impractical approach for two reasons:
1. It's not practically relevant. Many users will screw up profiles
instead of improving on them when they start to tweak them manually
(cf. what Bob Marchant wrote above). Editing profiles to get
metrologically better results needs a *lot* of experience and many
people will fail. It's not relevant what DNG profiles could achieve
theoretically, in the hand of an absolut expert; it's relevant what
they can achieve in practice.
2. You are certainly aware that there are also profile editors for ICC
profiles. If I try to improve on Adobe profiles by tweaking them, I'll
have to do the same with ICC profiles. Basically, this is like Achill
(Adobe) and the tortoise (ICC): the tortoise is leading. Yes, Achill
can try and get better, but so can the tortoise. As you know, Achill
will never surpass the tortoise. :-)
You seem to argue like a car manufacturer who is confronted with
complaints that their car doesn't run fast enough, and replies by
pointing out the availability of tuning kits. Can the car be tuned?
Yes. Is this what the customer takes into account when talking about
the performance of a specific car? No.
You only tested to see if they matched the software they were
intended to match - and they did.
You stated that Adobe's DNG Profile technology and methodology do
not exceed the quality of ICC profiles, which is something you
failed to test.
I really don't know how I could make my conclusions any clearer for
you. Of course the point isn't that the Canon Emulation profiles
aren't better than the original Canon profiles. The point is that
Adobe's own Standard profiles aren't better.
You know, Adobe isn't the hub of the universe although some people
here certainly behave as if it was. I cannot spend hours on one
single profile to see whether I could improve on Adobe's default if
I have to assess much more than 100 profiles each of which would
have to be treated in the same way (i.e. also tweaked to improve on
it). I simply have to assume that each vendor tried to deliver by
default the best they could deliver. This is common practice in
reviews such as this.
If stating incorrect and unsupported conclusions is standard
practice for your publication, then it needs to be labeled as a
marketing document and not a review.
Why is it so hard for you to stay away from this kind of ad-hominem
arguments? By your standards, only marketing documents (marketing for
what, BTW?) would exist in this world. There is not a single
scientific experiment that can avoid assumptions about ancillary
conditions.
Why is it unreasonable to assume that Adobe delivers default settings
as good as possible for them?
My testing allows me to state the following:
1. Adobe's "Canon Faithful" emulation profile matches the original
profile within a small delta E in my 7 test lighting scenarios (not a
bad result at all ...)
2. Adobe's "Standard beta" profile fares worse metrologically than
the
"Canon Faithful" emulation profile.
That is odd, since the Canon (and Nikon) default renderings are
known to be
far from accurate (high contrast, high saturation, selective color
saturation boosts, etc.).
They aren't accurate. But neither is Adobe's Standard profile.
3. The (original) Canon "Faithful" profile is the metrologically
best profile of all profiles Canon delivers for my test camera.
I'm starting to wonder about your standard of comparison here....
Why? Obviously, one of the 6 Canon profiles must be the best Canon
profile. Why is it so unbelievable that it's the "Faithful" one?
Especially since Canon itself writes in its manual that this is the
profile designed to be metrologically correct.
Now I rely on 2 assumptions:
6. Adobe's emulation of original Canon profiles other than
"Faithful" are similarly close to their originals as the "Canon
Faithful" emulation is
7. Adobe tried to deliver the best default settings they could for
their "Standard beta" profile as well as for profiles resulting
from measurements of a ColorChecker 24 target with the DNG Profile
Editor.
(Again, a comparative review has to make this assumption.)
You cannot make assumption 6 or 7.
Why not 6? Are you saying that Adobe tries to emulate "Faithful" as
good as possible, but has no intention of matching the other of
Canon's defaults as well? Why should Adobe behave this irrational?
As to 7, I have to make this assumption. Every review I ever came
across made it. Otherwise the first review ever would be published in
a hundred years. Sorry, if this still isn't clear to you, I don't know
what more I could possibly say. Our whole economies are based on the
assumption that corporations act rationally.
You have no idea what Adobe's targets were for those profiles.
No, but why should I care?
So the conclusions from my testing are:
8. from 1 & 3 & 6 -> Adobe's "Canon Faithful" emulation profile is
the
best of all of Adobe's Canon emulation profiles for my test camera
9. from 8 & 2 & 5 -> Adobe's "Canon Faithful" emulation profile is
the
best of all of Adobe's profiles for my test camera
"best" only if your target is Canon's default rendering.
It is not the most accurate by far.
But Adobe's other profiles were worse.
10. from 9 & 1 & 4 -> Custom profiles built with ProfileMaker,
Digital
ColorChecker SG and Capture One PRO deliver better metrological
results than the best of all of Adobe's profiles for my test camera
9) is what I meant when I wrote that "Adobe, with its proprietary
technology, still can't improve on the best Canon factory profile".
No, that statement does not follow from your tests.
OK, here is the point were arguing simply doesn't make sense anymore.
Are you saying that if someone does not agree with Adobe's POV, he
must be either a troll or stupid?
No, but that you may not have the background to understand where you
are falling short in your analysis and conclusions.
Which is the "politically correct" way of calling somebody stupid ...
I'm doing this because I conducted measurements and they revealed
Adobe profiles, in their current implementation and within a
practical usage scenario, to be metrologically inferior to custom
built ICC profiles. I got numbers, and I reported them. It's called
science.
No, you tested a few profiles, not the profile technology.
You cannot test a "technology", you can only test specific
manifestations of it. Which is what I did.
Bye
Uli
________________________________________________________
Uli Zappe, Solmsstraße 5, D-65189 Wiesbaden, Germany
http://www.ritual.org
Fon: +49-700-ULIZAPPE
Fax: +49-700-ZAPPEFAX
________________________________________________________
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Colorsync-users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden