Starting a new thread to avoid hijacking another, unrelated thread about picking a camera ...
I'm very curious how Andrew Rooney (aka "The Digital Dog") tests/evaluates the quality of profiles.
Everything was visual. Lots and lots of real world and synthetic images. Some of Bill Atkinson's test images are wonderful (28 Balls). The 28 Balls image is a ball breaker and shows a lot about the color engines. The Roman 16's are always used.
Is this test/evaluation something that a well calibrated X-rite i1iSis could do "by the numbers" with "apples and apples" De2k's using the X-Rite MeasureTool's Compare freeware (2.0.# on a PC, 1.5.# on a Mac)?
Hypothetically, suppose someone claimed that profiles from ArgyllCms are typically as good or better than those from ProfileMaker-5, and almost as good as i1Profiler 1.4.2. Is that something that could be objectively measured by an automated spectro, rather than an informed, but still subjective, opinion? * Could there be some kind of weighted "figure of merit" of, for example, De2k + Adjusted-Gamut-Volume? * Are De2k accuracy + gamut-volume the primary considerations when evaluating profile quality? What else? Banding? Is there a way to measure that, and reduce to a number grade by a machine? * How important is it to have "flexibility" so that the same profile to handle more situations? Or would a pro printer be inclined to use multiple separate profiles for the same printer + paper + ink for different photos ... like portraits vs landscapes? (ignoring b/w which I have no experience with) * Depending on photo and customer, should flesh tones be given extra weight in a "De2k + gamut-volume based figure of merit"? * Should "Intent" be a consideration: Perceptual vs RelativeColor vs Saturation vs Absolute? * What are characteristics of a series of good test prints? Lots of gamut vs low-gamut vs reasonable gamut? * I would think that one profile might be better for flesh tones, but not so good with synthetics. One profile might "win" 4 out of 8 print tests, but the other profile would "win" flesh tones, and they would "tie" on the other three. Which profile "wins"? Depends? ... * For those with access to an i1iSis, how useful would it be to have aligned i1iSis patches integrated into a test print? The top part of the letter sized test-print would be about 150 to 1000+ carefully chosen patches, and the separately printed bottom part could be "28 balls" or "Roman 16". Or the bottom half was a series of human "Shirleys" with carefully exposed, non-synthetic flesh tones? Or i1iSis patches on the equivalent of the OutbackPhoto test print? ... * Obviously, the correct answer to many or most of the above questions would be: "it depends". * Here are some examples that this admittedly CM newbie has been trying out: http://berean.zenfolio.com/isis_test_prints http://berean.zenfolio.com/isis_test_prints/h5ddc538a#h5ddc53b6 ... * What if the evaluation broke out different "categories" of patches, like Thomas Fors / Rags Gardner's AcrCalibrator freeware .jsx scripts do for camera profiles: ** blacks and dark grays, mid grays, light grays and white ** dark reds, mid-reds, saturated reds ** dark blues, mid-blues, saturated blues, multiple sky blues, ** dark greens, mid-greens, saturated greens, multiple grass greens ** etc ... ** orange, cyan, pink, violet/purple, periwinkle, "sky purple", etc. ** light flesh tones, mid flesh-tones, dark flesh-tones, by continent? ** etc, etc.