25 Replies Latest reply on Nov 4, 2008 11:11 PM by SimonTindemans

    How to shoot Color Checker for DNG profile editor?

      I'm trying to learn and understand the calibration process using the Macbeth Color Checker. Spent time months ago with the Fors/Tinderman/Rags scripts, and now the new DNG Profile Editor.

      I remain a little confused about how to shoot the color checker. I've made tons of bracketed shots under various light conditons. Direct sun, bright shade, studio lights, etc. But in all my Color Checker images, the grayscale tone values vary significantly from the target values. When I pick an exposure whose white square (as read in ACR) is near the target value of 243 to 245, the black square is also close to the target value. But the four middle gray squares are always high. About 15 to 30 points high.

      How important is it to have a base exposure of the Color Checker that matches the Macbeth grayscale target values?

      Profiles I created using the chart wizard in the DNG profile editor seem to generate reasonable results on real world images, but the Color Checker images made with these custom profiles have color values that vary quite a bit from target values. If I build a custom tone curve on one of these images and force all the grayscale squares to target values, the color squares move closer to their target values, but are still quite a bit off.

      So I'm still confused. In addition to shooting the Color Checker as close to 6500K as possible, do the grayscale values need to line up, and if so, how do you do that?
        • 1. Re: How to shoot Color Checker for DNG profile editor?
          MadManChan2000 Adobe Employee
          Hi Kelly, in general the tonality/exposure does not matter, because the PE's chart wizard only adjusts hue and saturation, not lightness. This is mostly due to our belief that the profile should get the basic hue & saturation right, but leave control over tonality up to user control, not profile control. (As an aside, I'll note that you can change the default tone curve of the profile by using the 2nd tab.) So, in short, PE makes no attempt to match the gray patches to the overall reference levels of the chart.

          The main thing to avoid when shooting the chart is overexposure or underexposure. By bracketing you should definitely avoid this, i.e., you'll end up with at least 1 good frame. PE will do some sanity checking to make sure you don't have clipped channels and will warn you if a particular patch in your image is suspect.
          • 2. Re: How to shoot Color Checker for DNG profile editor?
            michael shaffer Level 1
            I don't know if it's a bug or not, but I'me still getting error messages that imply the 'white' patch "has a significant color cast".

            Yesterday, I waited most of the day here in Newfoundland for the sun and blue sky to emerge. For 2 images, all things equal and seconds apart, one of them reported the error and the other did not. I don't think it was noise because the camera is an Oly E-3 shot at ISO100.
            • 3. Re: How to shoot Color Checker for DNG profile editor?
              MadManChan2000 Adobe Employee
              I will probably twiddle with the tolerance setting for the white patch.

              That said, in the meantime please go to the first tab (shortcut: ctrl/cmd-1) and move the cursor over the white patch in the image. The RGB readout will give you the numbers for the white patch. What are they?

              Sometimes you can get the PE to be more agreeable by moving the white marker (the white circle) around the white patch (i.e., nudge it slightly).
              • 4. Re: How to shoot Color Checker for DNG profile editor?
                michael shaffer Level 1
                > please go to the first tab (shortcut: ctrl/cmd-1) and move the cursor over the white patch in the image. The RGB readout will give you the numbers for the white patch. What are they?

                They bounce but here are several 'white' patch values (R,G,B - noting truncation issue - ie, put a '1' in front of each):
                93,87,82
                95,89,83
                93,87,80
                94,88,82

                Also of note, relevant or not, is that I notice some variation of my images' whitepoint values relative to what I'd expect (patch R4C2). For example (and I realize there are reasons for small differences), one of my MCC images (which also fails the "significant color cast" tolerance), shot under cloud cover, would lead me to believe that its Lr2 eye-dropper analysed R4C2 value should be nearer 6000k, rather than the "as shot" value of '4900/5' or the actual eyedropper calculation, '5150/3' (which also happens to be the Wb as when loaded by DNG PE).

                Any thoughts as to difference between what was expected v reported and its possible effect on PE outcomes? Should I look into it further, or simply suspect something akin to "cloud cover at high latitudes"?
                • 5. Re: How to shoot Color Checker for DNG profile editor?
                  G'day Eric,

                  Could you please explain the rationale for not also including the tonality in the profile? Surely it is just as important to get that accurate as it is to get colours. Personally, I would prefer to have the profile get everything accurate by default but I would still have the opportunity to adjust the tones in Lightroom, anyway.

                  If I was to use the second tab to set the tone curve, how do I know that I have it correct? What do the numbers for RGB actually represent? Are they percentages like in Lightroom, or are they 0-255 like most others? If they are percentages, is it linear i.e. you could calculate the value, or something else?
                  • 6. Re: How to shoot Color Checker for DNG profile editor?
                    MadManChan2000 Adobe Employee
                    Andrew, I am assuming you are referring to profiles built with the Chart Wizard feature of the DNG Profile Editor. No tone curve is required for such profiles, since these profiles are designed with scene-referred colorimetry in mind, i.e., for raw data which is still linear. The profiles built by the Chart Wizard, however, use the default Camera Raw tone curve because most photographers prefer the visual results compared to a strictly linear tone curve (which tends to make images look flat).

                    To make this really concrete ... let's say you're trying to "go by the numbers" to get your images as "accurate" as possible. Let's go through a little exercise; part of the point of this exercise, however, is to convince you that "accurate" tonality is not really what you want, at the end of the day.

                    The ColorChecker reference numbers of the 6 gray patches when mapped to 8-bit ProPhoto RGB would be 240, 190, 145, 105, 67, 37 (or something like that; I'm going by memory).

                    Now let's say you've taken a picture of this ColorChecker and you've taken care to make sure it's uniformly illuminated. In Camera Raw, make sure your working space is set to ProPhoto RGB (via the workflow options near the bottom of the dialog box).

                    Then, just as an exercise, try setting Brightness, Contrast, and Blacks to 0, set Point Curve to Linear, and leave everything else at defaults; this gets you into "linear" mode. Do a click-WB on the 2nd patch in the 4th row. Use the samplers tool to set samplers on each of the 6 gray patches in the 4th row. Adjust the Exposure slider till the sampler for the 2nd gray patch reads 190. Then check the other samplers: they should pretty much match the numbers I listed above.

                    This is essentially "linear" processing, without any tone curve, and it will match the ColorChecker in tonality. But in general images don't look good with this setting.

                    So my recommendation is not to get too hung up on the notion of getting "accurate" tonality with a profile. The profile's primary contribution is getting the relative hues & saturation right, but ultimately tonality should really be under user control, rather than profile control.
                    • 7. Re: How to shoot Color Checker for DNG profile editor?
                      Level 1
                      G'day Eric,

                      Thanks for your reply. I'm not sure I can do the exercise as I only have Lightroom and the DNG Profile Editor. I don't have Photoshop which (I believe) means I don't have a standalone Camera Raw, only what is built into Lightroom.

                      The default Camera Raw tone curve is still too flat for me and I'd have to modify the tone curve on every single image. What I'm looking for is to get a curve that is closer to the end point than what I have now. If that means it is not "by the numbers accurate", I'm OK.

                      While I have your attention, why is it when I load my DNG image of the ColorChecker into PE and run the chart wizard the chart has good contrast. If I then create a profile and use that profile in Lightroom to convert the exact same image or the NEF from which it was created, the image looks flat? Shouldn't they look the same?
                      • 8. Re: How to shoot Color Checker for DNG profile editor?
                        MadManChan2000 Adobe Employee
                        Andrew, all of the comments I made regarding Camera Raw / Photoshop are find with Lightroom as well, just that the numbers are presented differently. In particular, you have access to the same set of adjustments I described.

                        You absolutely do not need to make tone curve tweaks separately for each image. There are several ways to simplify this. (1) Make your desired tone curve, and save it as part of your default settings for your camera. (2) Edit the profile using DNG Profile Editor; use the 2nd tab (tone curve) to modify the tone curve to make it look the way you want. Save the modified profile, and make this profile the default for your camera.

                        For your last paragraph, they should look the same, yes. I am guessing that you somehow changed the image settings in LR. Maybe you can post a screenshot showing your Develop settings for the NEF within LR.
                        • 9. Re: How to shoot Color Checker for DNG profile editor?
                          Level 1
                          Here is a link to a screen capture of the whole Lightroom screen with the image of the ColorChecker chart loaded and the profile I created in the Profile Editor in use:

                          http://andrewtrevor-jones.com/LightroomDevelopSettings.jpg

                          Here are the rest of the develop settings:

                          http://andrewtrevor-jones.com/LightroomDevelopSettings2.jpg

                          Here is the same image in the DNG Profile Editor after running the chart wizard. This is using the default tone curve.

                          http://andrewtrevor-jones.com/DNGProfileEditor.jpg

                          You can see that the file in Lightroom looks flat compare to how it looks in the profile editor.
                          • 10. Re: How to shoot Color Checker for DNG profile editor?
                            Jao vdL Adobe Community Professional & MVP
                            You should be using the medium contrast point curve (i.e. the default) in Lightroom to make the images match. Also Brightness and contrast should be at their defaults of 50 and 25 and blacks at 5. I also see you moved the sharpening and noise reduction controls from their defaults but those should not matter for color and contrast.
                            • 11. Re: How to shoot Color Checker for DNG profile editor?
                              Level 1
                              I haven't actually changed any of the develop settings. That's how they all were by default. Is there something wrong with my installation? i.e. everything was zero by default.
                              • 12. Re: How to shoot Color Checker for DNG profile editor?
                                Jao vdL Adobe Community Professional & MVP
                                The history shows you applied a preset upon import - zeroed, which zeroes out all develop controls. You probably set that in the import panel at some point and Lightroom is remembering this. Just hit reset, reselect your profile and see your glorious colors return. Also next time you import pay attention to the settings.
                                • 13. Re: How to shoot Color Checker for DNG profile editor?
                                  Level 1
                                  I have been using "Import/General - Zeroed" since LR 1.3 because the defaults were way over the top - too much contrast, too much brightness. Perhaps they are more appropriate now when using the new profiles.

                                  I did reset and they did not come back. I even did Alt-Reset and selected "Restore Adobe Default settings" and they are all still zero.
                                  • 14. Re: How to shoot Color Checker for DNG profile editor?
                                    Jao vdL Adobe Community Professional & MVP
                                    Reset should have reset it to the default settings I noted above. Perhaps you have defined a camera default. To clear those, go to Preferences->Presets and hit "Reset all Default Develop Settings" and try the reset again. I'm surprised you find the default settings too contrasty. For my D300 and my taste they always need some extra contrast as I find the default a little too flat.
                                    • 15. Re: How to shoot Color Checker for DNG profile editor?
                                      MadManChan2000 Adobe Employee
                                      Andrew, Jao explained the situation. Basically CR/LR is setup so that they will give you the profile's base adjustments when at their default settings (brightness 50, contrast 25, etc.). If you adjust any of these sliders then all bets are off.

                                      Of course, adjusting them in practice will be useful on an image by image basis. That's the whole point of rendering a raw file under user control.

                                      But the profile baseline is at the CR/LR defaults.
                                      • 16. Re: How to shoot Color Checker for DNG profile editor?
                                        Level 1
                                        OK, I have managed to get back to the LR defaults. I now know why I went with zeros - I really don't like the LR defaults. I guess everyone is different.

                                        I have managed to set up a tone curve in LR which when combined with the profiles, give me the results I want. I fact, if I use the "Camera Neutral Beta 2" profile and the new tone curve, the converted NEF is almost identical to the JPEG which is something I was trying to achieve (and I know a lot of other people would like to do). Note that I'm not saying that the JPEG is "correct" or that it will be my end point, I just like it as a starting point. I now also have the option of switching to the profile I created myself from the ColorChecker so I can choose between "Nikon colours" and "accurate colours".

                                        I now better understand the profiles (and also when when I first tried the new beta profiles why I didn't like them). Thanks for the help.
                                        • 17. Re: How to shoot Color Checker for DNG profile editor?
                                          Jao vdL Adobe Community Professional & MVP
                                          Andrew, the profiles are designed to give you almost exactly the jpeg rendering when you make ALL settings default. No need to play with custom curves. I don't know how Eric did this but I am guessing he simply measured from jpegs that came out of the camera.
                                          • 18. Re: How to shoot Color Checker for DNG profile editor?
                                            Level 1
                                            Well there you go. I have been running with everything zeroed for so long no wonder I couldn't get the profiles to work.

                                            I reset back to defaults and applied "Camera Neutral Beta 2". The NEF is almost indistinguishable from the JPEG. This is great. Now I know why the profiles have been received so well. And I still have the choice between "Nikon Colours" and "Accurate Colours" by virtue of my own profile.

                                            Thanks.
                                            • 19. Re: How to shoot Color Checker for DNG profile editor?
                                              SimonTindemans Level 1
                                              Hi Eric,

                                              Thanks for the beta 2 update. At the risk of sounding like a broken record, let me reiterate my main request for the DNG profile editor.

                                              As you said, the chart wizard currently assumes that the resulting profile will be used in conjunction with the ACR default tone curve + blacks setting. Although I agree with your statement that most people will prefer this over a truly linear curve, I still think that there is a fair number of photographers that has a preference that differs from the ACR defaults. Personally, I think blacks=5 is a very unpleasant choice for many images, and I occasionally have a need for a truly scene-referred (linear) image as well [stitching, HDR, alternative processing].

                                              I would be really useful if the Chart Wizard would offer an additional option for those people who wish to use a different tone curve. This could be implemented in two (or more) ways:

                                              1) An *option* to create a scene-referred profile that leads to an exact ColorChecker match when used with a linear tone curve. Using such a profile with the ACR default curve would of course lead to an increased saturation across most of the range and a slight desaturation in the highlights.

                                              2) An option to use the tone curve from the embedded DNG conversion settings and create a profile to work with those settings. Preferably, this tone curve would simultaneously be copied into the profile tone curve. This would allow people to select their preferred tone curve in ACR and create a profile with the same tonality. One catch would be that you'd need to decide how much additional saturation would be preferred by people. This could be done by interpolating between the two obvious benchmarks: 0 (accurate) for scene-referred and whatever the current amount is for ACR defaults.

                                              Regards,
                                              Simon
                                              • 20. Re: How to shoot Color Checker for DNG profile editor?
                                                MadManChan2000 Adobe Employee
                                                Simon, I did hear your request. In fact, beta 2 of the DNG Profile Editor already fulfills #1.

                                                Sounds like #2 is a request for a mix of scene-referred & output-referred profile.
                                                • 21. Re: How to shoot Color Checker for DNG profile editor?
                                                  SimonTindemans Level 1
                                                  Hi Eric,

                                                  Sorry, I have looked for such a feature after v2 became available and haven't been able to find it. I've searched for a switch and I've tried to see whether there were any differences when the Chart Wizard was run with a linear base curve (adjusted afterwards) and the ACR default curve. No results so far... Can you point me in the right direction?

                                                  In my testing I did come across a few small things, though:
                                                  1) In the tone curve tab, choosing the ACR default curve gives a *much* darker image than selecting the linear curve and manually reconstructing the ACR default curve as shown by 'Show Base Tone Curve'. Is that by design? [maybe one is ~gamma2.2-corrected and the other isn't]

                                                  2) In the truly linear / scene-referred profile building mode (that I can't find), do you also adjust for blacks=0? I hope so, because blacks=5 is a very dominant correction.

                                                  3) I still find it very confusing that the profiles should be combined with the default curve settings in ACR. For example, when creating a linear profile, the 'medium contrast' tone curve should be selected in ACR in combination with brightness=50 and contrast=25. It would be much more intuitive if these were all reset to zero (like when working on JPGs). This would at least indicate that you are working 'on top of' some other set of defaults. Also, as an example, using brightness=50 and contrast=25 in combination with a 'linear' profile would then produce a properly toned image.

                                                  Of course, there is a lot of inertia because 50/25/medium has been the default for so long, but I'm afraid the current format will only lead to more confusion in the long term.

                                                  Finally, my no.2 request would indeed lead to a mix between scene-referred and output-referred profiles. In a way, that's not very different from the current default Chart Wizard profiles. They are based on a scene-referred measurement, but adjusted to bring the in line with output expectations. My proposal would be to interpolate the ingredients of this mix, starting from the two 100% scene-referred and the 'ACR-default' settings. Admittedly, having two options already goes a long way towards satisfying the needs of diverse users.

                                                  Simon
                                                  • 22. Re: How to shoot Color Checker for DNG profile editor?
                                                    MadManChan2000 Adobe Employee
                                                    > Sorry, I have looked for such a feature after v2 became available
                                                    > and haven't been able to find it. I've searched for a switch and
                                                    > I've tried to see whether there were any differences when the Chart
                                                    > Wizard was run with a linear base curve (adjusted afterwards) and
                                                    > the ACR default curve. No results so far... Can you point me in the
                                                    > right direction?

                                                    There is no switch in the UI (a good reason you didn't find one!).

                                                    A change was made to the Chart Wizard's profile generation routines. With the change, DNG Profile Editor beta 2 will produce (reasonably) accurate scene-referred profiles.

                                                    However, you should only expect to see numbers that match the chart's reference numbers if you're using a truly linear tone curve. You can do this in two ways. (1) Use the 2nd tab of the DNG PE to set the base tone curve to Linear, and don't adjust the curve pane, and leave the tone curve within CR/LR at the defaults; or (2) leave the profile's tone curve alone and instead adjust CR/LR to have a linear tone curve.

                                                    > 1) In the tone curve tab, choosing the ACR default curve gives a
                                                    > *much* darker image than selecting the linear curve and manually
                                                    > reconstructing the ACR default curve as shown by 'Show Base Tone
                                                    > Curve'. Is that by design? [maybe one is ~gamma2.2-corrected and the
                                                    > other isn't]

                                                    The user tone curve in PE is operating in a gamma 2.2 space, not linear. The base tone curve in PE is plotted in a linear space. Hence the difference.

                                                    > 2) In the truly linear / scene-referred profile building mode (that
                                                    > I can't find), do you also adjust for blacks=0? I hope so, because
                                                    > blacks=5 is a very dominant correction.

                                                    Yes.

                                                    > 3) I still find it very confusing that the profiles should be
                                                    > combined with the default curve settings in ACR.

                                                    I hear you. On the other hand, it's probably easiest for most users since most users start with the CR/LR defaults.
                                                    • 23. Re: How to shoot Color Checker for DNG profile editor?
                                                      SimonTindemans Level 1
                                                      Ahh...good news. Do I understand correctly that the Chart Wizard now *always* aims for a scene-referred match (no tone curve, blacks=0) and then applies the tone curve, i.e. no special saturation compensation takes place? Is the color matching always done with blacks=0? [just double checking here]

                                                      >The user tone curve in PE is operating in a gamma 2.2 space, not linear. The base tone curve in PE is plotted in a linear space. Hence the difference.

                                                      It would be nice if the base tone curve were also plotted in ~gamma 2.2. That way, you can manually 'copy' a base curve and change the base to linear to start adjusting from the default curve. Admittedly, this is a minor niggle.

                                                      Thanks!
                                                      Simon
                                                      • 24. Re: How to shoot Color Checker for DNG profile editor?
                                                        MadManChan2000 Adobe Employee
                                                        Hi Simon, yes, the Chart Wizard always aims for a scene-referred match, no saturation side effects.

                                                        Of course, the actual profile that gets built (assuming you don't tweak anything) will still use the default CR tone curve if you use the profile within CR with CR's defaults. However, if you choose to use a linear tone curve (using either one of methods I described above) then you will really be getting scene-referred results.
                                                        • 25. Re: How to shoot Color Checker for DNG profile editor?
                                                          SimonTindemans Level 1
                                                          Great! Thanks for clearing that up.
                                                          Simon