Re: Media Testing for maclife.de
Re: Media Testing for maclife.de
- Subject: Re: Media Testing for maclife.de
- From: Chris Cox <email@hidden>
- Date: Wed, 10 Sep 2008 17:55:50 -0700
- Thread-topic: Media Testing for maclife.de
On 9/9/08 9:10 PM, "Uli Zappe" <email@hidden> wrote:
> Am 10.09.2008 um 02:36 schrieb Chris Cox:
>
>>> They can (meaning they provide better results than e.g. Adobe's
>>> dual approach default profiles in *every single* lighting condition
>>> from bright sunlight to a halogen lamp in a dark room) - that was
>>> one of the more interesting results of my test, and I reported it
>>> here. Did you actually read anything of my review summary?
>>
>> A single ICC profile cannot account for multiple lighting conditions
>> - that is one of the big, well known, well understood limitations of
>> ICC profiles.
>
> And wrong, as my measurements revealed (in the meaning I explained
> above).
I'm sorry, but that is a fundamental limitation of ICC profiles.
If you are using ICC Profiles with multiple light sources and getting
anything near accurate results -- then something else is involved and
handling corrections for the white point/light source. And you haven't
documented what else was done to account for varying light sources.
> Of course, this result is remarkable, but the numbers are clear: my
> best ICC profile was better in *any single* lighting condition than
> e.g. Adobe's default profiles. See my upcoming report, where I will
> publish the numbers.
Yes, it would be remarkable, if it were possible.
But alas, it is not -- without some other process involved.
And to retain credibility, you need to document that process.
> Another thing is to explain these surprising numbers.
>
> Let me point out that Capture One PRO (which I used for my ICC
> profiles) performs the white balance adjustment *before* the ICC
> profile assignment.
Ah, so there was another process involved.
And you are not using the profiles for calibration at all, but only for
artistic intent and post-process color correction.
> It's clear from this processing sequence that the
> ICC profile has no way to compensate for peculiarities of the spectral
> response of the camera sensor, since after the white balance
> adjustment, these have "moved".
Which means you are not profiling the camera at all, but rather profiling
the RAW conversion software and making corrections/adjustments to the
processed RAW output instead of the camera RAW data.
> So what is it that the ICC profile improves? The only explanation I
> have that it's something like (for lack of a better word) a general
> "color character" of a camera, e.g. the prevalence of a certain color,
> no matter under which lighting.
You improved the rendering of the RAW package to match whatever you
expected. Yes, ICC profiles can be used for color correction, and yes, if
the processing is decent they will do a good job of matching your target.
Unfortunately, that is not what you set out to test, nor what you claimed to
test.
>
> It is true that my ICC profile, built with a target shot at 6500 K,
> provides better results for an image shot in sunlight than an image
> shot in halogen lighting. *But* even for an image shot in halogen
> lighting, the results for an image with the ICC profile are still
> better than those for an image with the best Adobe profile I could get.
See above. You aren't comparing like quantities or processes.
>
> So, you might say that the ICC profile doesn't really account for
> multiple lighting conditions, it's just that the ICC profile is so
> much better all in all than an Adobe profile (for whatever reason)
> that even when it provides suboptimal results for lighting conditions
> it was not built for, it's still better than the Adobe profile.
No, that's not even close to what is happening here.
Your testing procedure is flawed, so you compared ICC profiles as
post-processing corrections, and Adobe DNG profiles as
processing/calibration (leaving out your correction).
>
> But even this isn't the whole story. To explore this further, I built
> two profiles with the DNG Profile Editor and a ColorChecker 24. I just
> used the target images with the DNG Profile Editor and made no
> additional adjustments. I built one single-matrix profile, using only
> one target image shot at 6500 K, and one dual-matrix profile, using
> two target images shot at 6500 K and 2850 K. I then tested the two
> profiles for images shot under various lighting conditions.
>
> Obviously, you would assume that the dual-matrix profile fared better
> on average; however, surprisingly, the opposite was the case (though
> not by much).
Again, you are not comparing like quantities.
>
> So I can only conclude that while the idea of dual-matrix profiles
> sounds nice in theory, it does not really work in practice, at least
> not in its current implementation.
No, you cannot conclude that.
In fact, you can't conclude much from your testing.
>
>> How did you account for the different light sources using a single
>> profile?
>
> Apart from the white balancing before the profile assignment, not at
> all.
That little part -- kind of invalidates your stated results.
>
>> Or were you using an output (artistic intent) profile
>
> What's an "artistic intent"? I've never heard that expression.
You were using ICC profiles to correct the output of the RAW converter after
it had applied some calibration data to do white balancing and possibly
other color calibration transformations. You were not correcting the camera
sensor data, you were correcting the output of someone else's calibration
for sensor data.
>
>> not a camera calibration profile?
>
> I used an ICC camera profile built by ProfileMaker.
But your process was flawed, your understanding of the processing flawed,
and your results nearly useless.
>>> If this is the case, then why do the best custom ICC profiles
>>> deliver better results metrologically than any of Adobe's profiles?
>>> (In *various* lighting scenarios!)
>>
>> I don't know how they could, since the ICC profiles cannot handle
>> multiple lighting conditions by themselves.
>
> See above.
>
>> Most likely you are not telling us something important about your
>> test procedure.
>
> I don't know what this could possibly be, but wait for my upcoming
> report where I will try to describe my procedure in exact detail
I would withhold that report until you correct your testing procedure or
account for all the processing stages.
>
>>> ??? Adobe profiles can also only be used with the Adobe RAW
>>> converter.
>>
>> No, they can be used with any RAW converter that implements the
>> (public) DNG 1.2 specification.
>
> And what are the RAW converters apart from Adobe that implement the
> DNG 1.2 specification?
Anyone that wants to.
But I don't know how many are shipping at this time.
The most you can say is "Adobe DNG profiles are compatible with any RAW
converter that implements DNG 1.2, but only a few are known to implement it
at this time."
Your initial statement is overly broad, which makes it incorrect. But that
is possibly just a translation error.
>>>> But you did no testing to see if DNG profiles could exceed the
>>>> quality/accuracy of the default rendering...
>>>
>>> You mean by individually tweaking them?
>>>
>>> If so, I already wrote why this approach is completely unfeasible
>>> in a
>>> comparative review of this scope.
>>
>> In which case you cannot claim the conclusions you stated.
>> You did not test the DNG profiles to see if they could exceed the
>> quality.
>
> This is an important point to get clear.
>
> You suggest I should try and *manually* improve on the quality of the
> DNG profiles Adobe delivers.
No, I suggest you only state conclusions supported by your testing, and that
you correctly document your testing.
> You seem to argue like a car manufacturer who is confronted with
> complaints that their car doesn't run fast enough, and replies by
> pointing out the availability of tuning kits. Can the car be tuned?
> Yes. Is this what the customer takes into account when talking about
> the performance of a specific car? No.
More like a car manufacturer confronted by a customer saying that the anchor
isn't working. The manufacturer looks confused and says "cars don't have
anchors, what are you talking about?" The customer insists that the car he
bought has an anchor, sails and a rudder - and the manufacturer starts to
wonder about the customer...
>> You only tested to see if they matched the software they were
>> intended to match - and they did.
>>
>> You stated that Adobe's DNG Profile technology and methodology do
>> not exceed the quality of ICC profiles, which is something you
>> failed to test.
>
> I really don't know how I could make my conclusions any clearer for
> you.
By stating something that is supported by your testing.
Your current statements are not supported by your testing.
> Of course the point isn't that the Canon Emulation profiles
> aren't better than the original Canon profiles. The point is that
> Adobe's own Standard profiles aren't better.
That is different from your previous statements in a very important way.
>>> You know, Adobe isn't the hub of the universe although some people
>>> here certainly behave as if it was. I cannot spend hours on one
>>> single profile to see whether I could improve on Adobe's default if
>>> I have to assess much more than 100 profiles each of which would
>>> have to be treated in the same way (i.e. also tweaked to improve on
>>> it). I simply have to assume that each vendor tried to deliver by
>>> default the best they could deliver. This is common practice in
>>> reviews such as this.
>>
>> If stating incorrect and unsupported conclusions is standard
>> practice for your publication, then it needs to be labeled as a
>> marketing document and not a review.
>
> Why is it so hard for you to stay away from this kind of ad-hominem
> arguments?
Because you are producing something with mistakes and misunderstandings.
And it is not an ad-hominem attack, just trying to explain the seriousness
of the situation to you.
> By your standards, only marketing documents (marketing for
> what, BTW?) would exist in this world. There is not a single
> scientific experiment that can avoid assumptions about ancillary
> conditions.
You aren't anywhere near the standards for a scientific publication.
You aren't even near the standards for advertising materials in many
countries (where claims must be verifiable).
> Why is it unreasonable to assume that Adobe delivers default settings
> as good as possible for them?
Because you don't know what the design goal was for those profiles.
You can state what those profiles do, you can compare them to other
profiles, but you cannot read the mind of the people who made them. Trying
to do so, then make further claims based on that assumption is just bad
practice.
Uli, you seem to mean well.
But it also seems that every time you post you explain additional mistakes
in your testing procedure and your understanding of the software and
technologies.
You don't seem to know quite enough about the topic to adequately conduct
testing and make the conclusions that you have made. No, this does not mean
that I think you are stupid -- just that you need to do a bit more study
before conducting such tests and trying to publish conclusions. It would
also help you greatly if, in the future, you asked questions instead of
making accusations.
Chris
P.S. my ipod contents include 3 Ships and Olias of Sunhillow.
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Colorsync-users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden