33 Replies Latest reply on Aug 4, 2008 2:10 PM by Tim Lookingbill

    DNG Profiles and the ACR Calibrator

    Selby Shanly Level 1
      In the past the Fors ACR Calibrator (and the manual procedure conceived by Bruce Fraser) was required to account for production differences between instances of the same camera model. If I wish to use the new Adobe Standard Profiles, should I then rerun the Calibrator? I realize I will have to update the code to handle the new ACR version number - I already did this for 4.4.1.
        • 1. Re: DNG Profiles and the ACR Calibrator
          Level 1
          The odds are pretty good that the old method of running calibrations for cameras will be mitigated unless you have a camera sample that is well outside the bell curve. And yes, Tom will need to update the script for 4.5 and yes, if you select a new profile you'll need to re-run the script with THAT profile.
          • 2. Re: DNG Profiles and the ACR Calibrator
            Selby Shanly Level 1
            Thanks - I'll go do that.
            • 3. Re: DNG Profiles and the ACR Calibrator
              Why would you still want to use the calibration script?
              • 4. Re: DNG Profiles and the ACR Calibrator
                Level 1
                Wow. My criticism of the lack of Picture Style profiles in CR has gone by the wayside. I just fiddled with some daytime Arizona landscapes shot with a 5D/17-40.

                I compared the images using the default Canon Standard and Landscape PSs in DPP 3.4.1.1 to the same images in CR4.5 set at ACR default with sharpening off and the Adobe beta Picture Styles.

                They were remarkable alike. Only at 200% could I clearly see the very slight differences of some pixels no doubt due to the different demosaicing techniques (correct term I hope).

                In fact on a stand of reeds about 100m from the camera Camera Raw produced the same colors as DPP but with less artifacts and less errors. I am extemely impressed with Adobe's efforts.
                • 5. Re: DNG Profiles and the ACR Calibrator
                  MadManChan2000 Adobe Employee
                  Martin, instead of using the previous calibration scripts you can now use the Chart Wizard feature of the DNG Profile Editor, which performs largely the same function (but is faster and also benefits from the lookup tables in DNG 1.2 profiles). Here are 2 tutorials explaining how to do this:

                  http://labs.adobe.com/wiki/index.php/DNG_Profiles:Editor#tutorial_cc1

                  http://labs.adobe.com/wiki/index.php/DNG_Profiles:Editor#tutorial_cc2
                  • 6. Re: DNG Profiles and the ACR Calibrator
                    Level 1
                    Eric, I know that but I wonder what the use of the calibration script is when the Editor is now available and does everything better?
                    • 7. Re: DNG Profiles and the ACR Calibrator
                      Thomas Knoll Level 2
                      Martin:

                      Eric basically said the calibration scripts are now obsolete, and you are much better off using the chart wizard feature of the profile editor.
                      • 8. Re: DNG Profiles and the ACR Calibrator
                        Marco N. Level 1
                        I don't think scripts are currently obsolete until we can use the real target color values (read by spectrophotometer) and until we know in what way compute the tung/D65 target value starting from spectral data.

                        Infact it seems to me that ACR don't use neither bradford (like PS?) nor CAT02 CA and only about D50 target values all software are in agreement starting from spectral data.

                        And it seems that one good calibration in daylight condition ensures accuracy in the most lightning condition
                        • 9. Re: DNG Profiles and the ACR Calibrator
                          MadManChan2000 Adobe Employee
                          Yes, it would be nice to be able to import your own data from your own chart. You have a point there, but I think you're overstating it for the following reasons: (1) tolerances across charts are pretty tight, unless a chart is damaged (I've looked at a lot of CC data), (2) you are far more likely to get variations in lighting that far outweigh any differences across charts, and (3) the DNG Profile Editor offers far more degrees of freedom than the older calibration scripts. To expand on this last point: some of the older scripts optimized for only 3 patches; others optimized for all color patches (allowing you to specify weights), but ultimately the color matrix only has 6 degrees of freedom, so it's impossible to really nail all the colors.

                          Most CR color math is based on ProPhoto RGB primaries, D50 reference illuminant (see DNG 1.2 spec for details).

                          The DNG Profile Editor (PE) is using spectral data for several ColorChecker charts, averaged together. If you build a profile using Chart Wizard for D65 lighting, all the color math is done with respect to the D65 reference illuminant, and the chromatic adaptation to D50 (the PCS reference illuminant) is done via the linear Bradford transform. Same goes for illuminant A. So the only case that PE doesn't handle right now is the one where you want to import your own illuminant data.

                          Your last point about 1 good calibration in daylight: well, that'll work fine for most flavors of daylight, but not so well for tungsten (and vice versa). Measured data indicates significant variance in response across these two extremes; the degree of variation does depend on the camera model, however.
                          • 10. Re: DNG Profiles and the ACR Calibrator
                            Bill_Janes Level 2
                            >The DNG Profile Editor (PE) is using spectral data for several ColorChecker charts, averaged together. If you build a profile using Chart Wizard for D65 lighting, all the color math is done with respect to the D65 reference illuminant

                            Eric,

                            Not many of us in the field have D65 simulators, but we do have natural daylight at midday in northern latitudes, which is reportedly what D65 is based on. With the old Fors script, I have used natural daylight under the above conditions for calibration, but this is usually listed as 5400K, not 6500K as in the PE. What do you suggest?
                            • 11. Re: DNG Profiles and the ACR Calibrator
                              MadManChan2000 Adobe Employee
                              Hi Bill, I would choose 'Both Color Tables' from the popup menu in the Chart tab. This will use the D50 spectral illuminant data when computing the ColorChecker's target reference values. Note that the resulting profile should work well across many flavors of daylight, but won't be optimized for low color-temp (e.g., tungsten) situations.

                              If you later decide you wish to have the same profile be optimized for tungsten lighting, too, then photograph the chart again under tungsten, then be sure to select 2850 K from the popup menu in the Chart tab. Run the Chart Wizard again on your tungsten image and now you'll have a single profile that interpolates between the daylight adjustments and the tungsten adjustments automatically based on the white balance of the image.

                              (It's explained in a bit more detail in tutorial 6 of the docs.)
                              • 12. Re: DNG Profiles and the ACR Calibrator
                                rasworth Level 1
                                Eric,

                                I've read and re-read the tutorials; #6 explains how to create the single profile good between 6500 and 2850, but if I understood your last post you were describing how to create such with arbitrary daylight and 2850 tungsten. How about a step by step description, I'm not sure of the exact sequence.

                                Richard Southworth
                                • 13. Re: DNG Profiles and the ACR Calibrator
                                  Bill_Janes Level 2
                                  Richard Southworth wrote:

                                  >I've read and re-read the tutorials; #6 explains how to create the single profile good between 6500 and 2850, but if I understood your last post you were describing how to create such with arbitrary daylight and 2850 tungsten. How about a step by step description, I'm not sure of the exact sequence.

                                  Eric,

                                  I've read your post and also studied tutorials 5 and 6, but am still uncertain about photographing the Color Checker. The instructions say D65 and 2850 K illuminant (e.g., standard tungsten bulb). I don't have access to a D65 source, but rather use sunlight and my lights are 3200K. On checking, I see a standard 100-watt incandescent tungsten lamp is 2865K. I gather that noon sunlight may be adequate for the 6500K.

                                  As to Richard's question, I infer that one merely makes the two profiles after photographing under the two conditions mentioned above and these profiles are stored and used for interpolation.

                                  Of course, as someone pointed out earlier, one would make custom profiles only if there is some problem with the profiles already supplied by Adobe. Thus far the Adobe profiles seem to work quite well with my D3 in limited testing. In particular, the landscape profile has improved some of my landscape shots. Thomas and the rest of you are to be congratulated on this excellent addition to our toolbox.
                                  • 14. Re: DNG Profiles and the ACR Calibrator
                                    MadManChan2000 Adobe Employee
                                    Thanks for the nice feedback, Bill.

                                    To clarify the point brought up by you and Richard: there are two basic ways to use the chart wizard feature.

                                    First, you can use it to build a profile optimized for one lighting condition. This is the more familiar case, esp. to those of you who have used the CR calibration scripts in the past. This mode is described in Tutorial 5, and is enabled by choosing the 'Both Color Tables' option from the popup in the Chart tab. What happens here is that a single set of color lookup table adjustments is created, which is applied regardless of the white balance of the image. You can use any illuminant when photographing the ColorChecker; doesn't have to be illuminant A or D65. Such a profile will work pretty well as long as your real images don't stray too far spectrally from the illuminant you used to shoot the CC.

                                    Second, you can use it to build a more general profile as described in tutorial 6, but you would need to shoot the CC in conditions as close as you can get to A and D65 for best results. (~D50 or ~D55 lighting as a substitute for D65 should work pretty well.) The DNG 1.2 profile format actually allows the two illuminants to be different than A and D65, but currently the DNG Profile Editor's chart wizard feature only supports these two.

                                    Bill, you should be able to shoot the checker in the manner you describe (i.e., under noon sunlight on a blue sky day, and also under the incandescent bulb) and have it work well.

                                    When I was testing this, I used a standard household incandescent (very close to illuminant A; I was curious and measured it with an Eye-One spectro) and then tried various flavors of daylight (including Solux bulbs, which aren't that close to D65) for the D65 "half" of the profile. The daylight portion didn't make that much difference.
                                    • 15. Re: DNG Profiles and the ACR Calibrator
                                      rasworth Level 1
                                      Eric,

                                      Think I'm getting there, read thru the DNG spec, noted the capability of using other (fixed) lighting sources for the Calibration Illuminant.

                                      Ok, let me play it back. Unfortunately, the high noon light in Austin, Texas in July measures out at 5150/5, pretty far away from D65. I believe D65 was built from average measurements of high noon light in Western Europe, not sure why it's so much "hotter" but so be it.

                                      So I should shoot one image of the CC in the "best" daylight I can obtain, and one under tungsten. Fire up the wizard, using "both colors" so that both tables in the profile are filled with the same data, and do so with the daylight image. Without leaving that recipe, wizard the tungsten image using "2850" and use its results to replace same in the recipe. So now I have a custom recipe to build profiles, based upon my specific camera/lens.

                                      And as Bill wrote, the standard included profile does well, much improved over the 4.4 version.

                                      Richard Southworth
                                      • 16. Re: DNG Profiles and the ACR Calibrator
                                        rasworth Level 1
                                        Eric,

                                        The following question borders on the nerdy side, but I'm trying to "cement" my understanding of the profiling process.

                                        I'm assuming that all recipe generation with the Profile Editor is actually delta'd from existing Adobe generated profiles. After running the wizard the base profile is identified as ColorChecker. Is this a separate camera independent profile or the camera specific beta profile? I'm guessing the latter, since I could not find such in the profiles folders.

                                        Thanks for the info,

                                        Richard Southworth
                                        • 17. Re: DNG Profiles and the ACR Calibrator
                                          Thomas Knoll Level 2
                                          D55 is typical direct sun.
                                          D65 is typical cloudy day.
                                          D75 is typical shade on sunny day.

                                          When you use the color wizard, my understanding is the existing HSL tables are discarded and only the base matrices of the starting profile are used. So it does not matter very much which profile you start from when using the color wizard.
                                          • 18. Re: DNG Profiles and the ACR Calibrator
                                            rasworth Level 1
                                            Thomas,

                                            Got it, thanks. Your definition of D65 makes more sense to me based upon personal experience; however, the "mid-day sun in Western Europe / Northern Europe" is the more commonly used definition (e.g., wickipedia). Perhaps several cloudy days were included in the average. If we ever see another cloudy day here in Austin I'll be sure to photograph the cc.

                                            Richard Southworth
                                            • 19. Re: DNG Profiles and the ACR Calibrator
                                              Bill_Janes Level 2
                                              Richard Southworth wrote:

                                              >Ok, let me play it back. Unfortunately, the high noon light in Austin, Texas in July measures out at 5150/5, pretty far away from D65. I believe D65 was built from average measurements of high noon light in Western Europe, not sure why it's so much "hotter" but so be it.
                                              Richard,

                                              You raise an interesting point. I found an interesting source concerning the qualities of sunlight and daylight. Sunlight consists of only the direct rays from the sun, whereas daylight includes skylight and sometimes reflected light from surrounding objects such as trees and buildings.

                                              One main determinant of the color temperature of daylight is the altitude of the sun above the horizon, which changes with the seasons,time of day and location. I calculated the altitude of the sun at the summer solstice for Austin, Chicago, and Paris (30.37N, 42.86N, 48.85N latitude) and got 83, 71 and 65 degrees respectively. Since the sun is so much higher in the sky in Austin, one would expect the color temperature at noon to be higher in Austin than in Paris. Perhaps you are measuring sunlight rather than daylight.

                                              Since (per Thomas Knoll) a typical cloudy day approximates D65, one could expose the Color Checker on such a day. It would be interesting to make exposures at various times of day and note the white balance in ACR.

                                              http://www.handprint.com/HP/WCL/color1.html#dayphases
                                              • 20. Re: DNG Profiles and the ACR Calibrator
                                                rasworth Level 1
                                                Bill,

                                                Definitely measuring reflected sunlight, with a reasonable angle between the illuminant and camera path to hopefully pick up diffused light instead of specular.

                                                Ran out when a random cloud passed over the sun, was able to get up to 5550 or so, probably have to wait awhile for 6500. Not sure about the sun angle factor; at sunset and sunrise the light is of course very "red", but I believe the spectral content actually goes bluer when the light passes thru more atmosphere up to a point before shifting to red, therefore high-noon in Paris may result in a higher temp. than Austin. Not completely sure about this, but in any event I'll wait for more clouds.

                                                Richard Southworth
                                                • 21. Re: DNG Profiles and the ACR Calibrator
                                                  Tim Lookingbill Level 1
                                                  I agree about the look of 6500K/D65 having the appearance of overcast midday. I lived about 100 miles from Richard in Kerrville, Texas a little over a year ago and was eyeballing the neutrality of a Samsung CRT during winter at noon. My i1 Display measured the CRT at exactly 6500K. It also visually matched exactly the overcast formless white clouds outside my window.

                                                  I've also noticed a slight pinkish yellow tint to white objects lit by direct sunlight. The D55 setting in ACR does give this look but with just a little too much orange, but quite easy to adjust for with the sliders.

                                                  The look of the color temp numbers really doesn't account for how light affects the colors of real objects at different times of the day. Just shoot a white limestone brick wall or rock lit by direct sunlight at 9 AM, 1 PM and 6 PM and it will go from looking slightly greenish yellow with yellowish gray shadows, to buttermilk white with navy blue tinted shadows to rusty yellow with brownish blue shadows.

                                                  All five Raw converters I've tried out gets it wrong while the rest of the colors look correct no matter what color temp slider adjust chosen though ACR is the closest. I end up having to adjust Lab a/b curves in combination with color temp adjusts.

                                                  I also just found out a couple of days ago my left eye sees color temp with a slight orange cast while the right sees slightly green. Wonderful!
                                                  • 22. Re: DNG Profiles and the ACR Calibrator
                                                    Bill_Janes Level 2
                                                    >Not sure about the sun angle factor; at sunset and sunrise the light is of course very "red", but I believe the spectral content actually goes bluer when the light passes thru more atmosphere up to a point before shifting to red, therefore high-noon in Paris may result in a higher temp. than Austin. Not completely sure about this, but in any event I'll wait for more clouds.

                                                    I'm not sure about the sun angle theory either, but I thought you might find this article interesting as it agrees with your observations; this oserver's location was Atlanta.

                                                    http://www.imaging-resource.com/ARTS/TESTS/SUNSOURCE2.HTM
                                                    • 23. Re: DNG Profiles and the ACR Calibrator
                                                      Level 1
                                                      With all due respect to Thomas, Bruce and others that actively participated in creation and development of the ACR Calibrator script, this new feature from Adobe is much more comfortable to use and gives better results. The colors I'm getting from my Canon 5D and CC calibration are very close to the 4.4 profile, but flesh tones and greens look more natural. Kudos to the ACR team.

                                                      I would guess that one use for these profiles would be to mimic film looks. Did Adobe thought about it?
                                                      • 24. Re: DNG Profiles and the ACR Calibrator
                                                        Level 1
                                                        >I would guess that one use for these profiles would be to mimic film looks. Did Adobe thought about it?

                                                        Yeah...pretty much. In fact the intent is to get 3rd parties to create creative stuff with the DNG Profile Editor.
                                                        • 25. Re: DNG Profiles and the ACR Calibrator
                                                          Eric, Thomas, and everyone on the team, I want to add another "Thanks" to the forum. For me, this is one of the most powerful tools you all have come out with recently. The Fors script did magic for my D70, but my D300 color has been harder to nail down. Being able to have this degree of color control integrated with Lightroom is beautiful. I know that there have been third-party tools that generate ICC profiles and all that, but I love the elegance of this solution, since it gives me an accurate, custom-calibrated starting point in LR or ACR.

                                                          At some point, I could see this as a button under the Camera Calibration section that would pull in an image of a color checker from Lightroom or ACR, run a calibration on it, and give me a profile that could be used for the rest of the batch. Granted, it probably would be overkill to do this for every shoot, but I would love to see calibration made easy and accessible for people who aren't super-knowlegeable about color. There's nothing like having a neutral starting point. As the DNG Profile Editor stands right now, though, it's saved me hours of frustrated tweaking and has gotten my skin tones better than they've ever been. I thoroughly appreciate it. Keep up the good work.
                                                          • 26. Re: DNG Profiles and the ACR Calibrator
                                                            Tim Lookingbill Level 1
                                                            Been playing around with PE off and on all day.

                                                            It's pretty much Photoshop Selective Color, Replace Color and Curve tool rolled into one adjustment in the form of a profile that allows embedding into a raw file. Freakin' awesome.

                                                            I can now edit colors locally and separate from white balance's global affects. No more cyan-ish greens just to get the rust colored hue out of sunlit concrete.

                                                            One request. Could a fuzziness slider be added to selective color adjusts similar to Replace Color in Photoshop to control roll off of affected colors?

                                                            Can these custom DNG profiles made and exported out of PE be used in CS2's ACR 3.7? If so what directory do you have to save them to in Mac OS 10.4.11?

                                                            Also there is about a 4 second delay between tweaks and its affect on the preview and zoom view changes. Overall quite slow performance on my G5 iMac with 2 gigs of ram.

                                                            Thanks so much for coming up with this great tool.
                                                            • 27. Re: DNG Profiles and the ACR Calibrator
                                                              MadManChan2000 Adobe Employee
                                                              Dirk, you're welcome. Thanks for the comments; they're appreciated.

                                                              Tim, it turns out that there is already a fuzziness or "smooth falloff" built in. If you use the Show Affected Colors feature in the Options menu, it will show you a visualization of the extent of a selected color adjustment (check the color wheel to see which parts light up). The falloff is implicitly determined by the placement of your color control points. This prevents harsh, unnatural transitions between colors.

                                                              No, these profiles cannot be used with versions of CR prior to 4.5. They can only be used with DNG 1.2-compatible raw converters, and the only ones at this moment are CR 4.5 and LR 2.0. CR 3.7, for instance, doesn't contain any of the color lookup table code required to support the new profiles.

                                                              Preview updates do slow down with the more edits you have. That said, you can try the 'Fast Redraw' option in the Options menu (option-F shortcut).
                                                              • 28. Re: DNG Profiles and the ACR Calibrator
                                                                Tim Lookingbill Level 1
                                                                Thanks, Eric.

                                                                Yeah, I have to admit the Show Affected Colors does reveal a better roll off than what I'ld been getting using Selective Color's fuzziness slider or even using Lab curve tweaks on a test image scene I use of a morning sunlit limestone brick wall. I can actually make it look just as my eye saw it using PE.

                                                                Every raw converter I've used wants to render it as a monochrome rusty orange when it should have yellowish buttermilk mids with slightly bluish yellow gray shadows and greenish gray mortar.

                                                                Of course I haven't ever made a custom profile for my Pentax K100D using the Fors Script and a CC chart, just winged it using the converter's tools. I've just ordered the CC chart to see if I get the same level of color detail using PE.

                                                                Folks reproducing fine art paintings to extreme exactness will probably go gah gah over this tool.

                                                                It looks like you guys must've busted your *** coming up with a tool with this level of precision. I'm just dropped jawed with every tweak.

                                                                I wonder if this is Adobe's attempt at dangling a carrot to get everyone to convert to DNG. If it is, it's working for me at least.
                                                                • 29. Re: DNG Profiles and the ACR Calibrator
                                                                  Level 1
                                                                  >I wonder if this is Adobe's attempt at dangling a carrot to get everyone to convert to DNG. If it is, it's working for me at least.

                                                                  While it's not uncommon to assume a variety of motives to "Adobe" for the the development of the technology that is produced, most people overlook the most common and simple answer. The engineers do what they do because they honestly think it's the right thing to do and they are good at doing the right thing. They are also avid photographers so they are consumers of their own engineering efforts. The fact that their results are useful for other purposes is really just gravy on top.
                                                                  • 30. Re: DNG Profiles and the ACR Calibrator
                                                                    michael shaffer Level 1
                                                                    > I wonder if this is Adobe's attempt at dangling a carrot to get everyone to convert to DNG.

                                                                    Realizing I'm following Jeff's post which acknowledges Adobe engineers being photographers too -- suffice to add there's no real reason to suspect Adobe's "dangling any kind of carrot" when you realize the DNG PE cannot appropriately work, or (IMHO) ethically should not work with proprietary raw files.
                                                                    • 31. Re: DNG Profiles and the ACR Calibrator
                                                                      Tim Lookingbill Level 1
                                                                      Frankly, the DNG 2.1 spec is the most useful because it will allow the use of these table based edits performed in PE on the embedded DNG profile to be read by competing raw converters as long as they are DNG 2.1 compliant. I guess that's not much of an incentive for Adobe to "hold a carrot over our heads" to get us to switch to DNG.

                                                                      Some may prefer another raw converter's demosiacing and sharpening algorithms over ACR's but want the color precision available through PE. Very altruistic on Adobe's part.
                                                                      • 32. Re: DNG Profiles and the ACR Calibrator
                                                                        MadManChan2000 Adobe Employee
                                                                        This is a key reason why we named it the DNG Profile Editor, not the Camera Raw Profile Editor, Adobe Color Profile Editor, Lightroom Profile Editor, or something like that. The profiles are tied to the DNG spec, not Camera Raw/Lightroom, therefore making them portable across converters that support the 1.2 spec.

                                                                        Since the DNG 1.2 SDK is freely available (including code for reading and writing profiles), individuals and companies are also able (and encouraged by us) to create tools that operate on these profiles. Of course, DNG PE is one tool to build profiles, but other folks could also build tools to make profiles (just like there are multiple tools out there in the real world to build printer profiles).
                                                                        • 33. Re: DNG Profiles and the ACR Calibrator
                                                                          Tim Lookingbill Level 1
                                                                          Sorry.

                                                                          Got the DNG spec version number swapped. It's DNG 1.2.

                                                                          The warrantee on my 48 year old brain must be going out or I'm becoming dyslexic.