20 Replies Latest reply on Apr 1, 2008 11:52 AM by Panoholic

    ACR Overexposure with Nikon D3

    Bill_Janes Level 2
      Exposure to the right (ETTR) is a standard technique that requires placing of the image highlights just short of clipping in the raw file. Such exposure is often judged by the histogram or blinking highlights in the camera preview of the image, but is best determined by examining the raw file, since the camera preview is derived from a JPEG image to which the camera settings (white balance, tone curve, etc) have been applied. When one brings the raw file into Camera Raw, the white balance setting is applied, but the other camera settings are ignored. If the ACR histogram shows clipping of the highlights and a negative exposure adjustment is needed to recover them, this usually indicates that the image was overexposed. However, with the D3, the default ACR tone curve places the highlights too high and clipping occurs in the rendered file even when the highlights in the raw file are intact.

      I exposed a Stouffer step wedge so that step 1 was just short of clipping with the Nikon D3 using 14 bit NEF and the results are shown in an Imatest plot using the built in DCRaw converter. The results are linear as expected.

      http://bjanes.smugmug.com/photos/271579079_BGUhM-O.png

      Next, I split off the the green1 channel of the raw file with Iris (a freeware astronomical program)and examined the results in ImageJ (a freeware program from the National Institutes of Health). This plot shows that step 1 is just short of clipping:
      http://bjanes.smugmug.com/photos/271460382_RqUdu-O.gif

      and this is further confirmed by the histogram:
      http://bjanes.smugmug.com/photos/271460379_qwjtR-O.gif

      However, when the file is brought into ACR, with the default tone curve, the highlights are clipped as shown:
      http://bjanes.smugmug.com/photos/271460363_jMYHL-O.png

      and -0.6 EV of exposure compensation is required to bring Step 1 in range:
      http://bjanes.smugmug.com/photos/271460357_vDqYZ-O.png

      Nikon Capture NX renders the highlights of this file correctly.:
      http://bjanes.smugmug.com/photos/271569001_ZJwqX-O.png

      The conclusion is that the default ACR tone curve for the D3 is too "hot" and indicates overexposure when none exists.

      Next, here is the camera histogram of the properly exposed image. The histogram shows some clipping which is not really present in the raw file and which is eliminated by exposing 0.3 EV less. The conclusion is that the camera histogram is slightly conservative.
      http://bjanes.smugmug.com/photos/271460374_mRjof-O.png
      http://bjanes.smugmug.com/photos/271460388_WDMpW-O.png

      The blinking highlight display is even more conservative and indicates overexposure by 0.6 EV:
      http://bjanes.smugmug.com/photos/271460368_iFDsP-O.png

      For optimum results with ETTR, it is best to perform your own tests to determine the accuracy of the in camera histogram and blinking highlights so that these aids may be used for optimum exposure. Since the camera tone curve affects the results, it is important to use the same tone curve each time. Furthermore, one should be aware that the ACR default tone curve is too hot and that one does not necessarily have to reduce exposure when a negative exposure compensation is needed to bring the highlights into range. This ACR overexposure has been noted by others, but I do not know if it occurs with all D3s. If so, then the default tone curve should be revised by Adobe.

      http://www.nikonians.org/forums/dcboard.php?az=show_topic&forum=147&topic_id=53935&mesg_id =53935&page=2
        • 1. Re: ACR Overexposure with Nikon D3
          Panoholic Level 2
          Bill,

          is your example 12-bit or 14-bit?
          • 2. Re: ACR Overexposure with Nikon D3
            MadManChan2000 Adobe Employee
            G Sch: 14-bit, as he mentioned in his post. Also note the white levels close to 16 k.

            Bill: what you are seeing are the differences between how NX and CR set the zero point for exposure compensation. As you've figured out, to get the CR to match NX, set CR's Exposure slider to -0.5 (you empirically found -0.6). If you prefer this setting, you can save it as your camera default for the D3.

            Of course there will still be tone curve / contrast differences between CR and NX because CR's tone curve is different from NX's Standard picture control.

            Eric
            • 3. Re: ACR Overexposure with Nikon D3
              Panoholic Level 2
              ACR believes the saturation level to be 15892 for all three channels (ACR does not differentiate between the channels - this is one of the errors of the DNG specification).

              The actual clipping levels are 16383 (red) and 16263 (blue). The green channels have a range of non-linearity from 15750; I have seen different starting levels with different cameras.
              (Note: the above is not true re the fake ISOs.)

              Therefor ACR does not really know if clipping occured or not.

              There are no particular tone curves for different cameras.
              • 4. Re: ACR Overexposure with Nikon D3
                Hudechrome-sd9sPI Level 2
                A practical question: How does one actually get any work done if we can't trust the measurements system of our cameras and have to nail bite the decisions?

                A bit of a flip question, nontheless, when I work, I don't want to be encumbered by worries over ettr. Did I go far enough? Too far? Now what!

                Oh God! :D
                • 5. Re: ACR Overexposure with Nikon D3
                  Bill_Janes Level 2
                  >Of course there will still be tone curve / contrast differences between CR and NX because CR's tone curve is different from NX's Standard picture control.

                  That is quite true. NX applies a strong S-curve. With the exposure correction and standard black point at 5, ACR rolls off the shadows markedly and has a much flatter tone curve. With the black point set to zero and exposure to -0.6, there is a mild S-curve.

                  NX Standard Picture Control:
                  http://bjanes.smugmug.com/photos/271817949_8TrD2-O.png

                  ACR Defaults plus Exposure -0.6:
                  http://bjanes.smugmug.com/photos/271817941_4wk36-O.png

                  ACR Defaults plus Exposure -0.6, black 0:
                  http://bjanes.smugmug.com/photos/271817936_PJrFq-O.png
                  • 6. Re: ACR Overexposure with Nikon D3
                    Bill_Janes Level 2
                    > A practical question: How does one actually get any work done if we can't trust the measurements system of our cameras and have to nail bite the decisions?

                    Personally, I use the camera histogram to judge exposure to the right. With the neutral camera settings I used, this gives me about 1/3 stop of headroom, which is close enough for my work. According to what I have read, other cameras give more headroom, which is why testing is necessary. Once you confirm the behavior of your camera, you can trust the measurements.
                    • 7. Re: ACR Overexposure with Nikon D3
                      Bill_Janes Level 2
                      >The actual clipping levels are 16383 (red) and 16263 (blue). The green channels have a range of non-linearity from 15750; I have seen different starting levels with different cameras.

                      Gabor,

                      Using your Rawanalyze tool, I get a value of about 15970 for my camera. The right digit of the display is cut off in the display as shown:

                      http://bjanes.smugmug.com/photos/271834950_3FhNB-O.gif
                      • 8. Re: ACR Overexposure with Nikon D3
                        Panoholic Level 2
                        Bill,

                        the difference between 15970 and 15750 seems really irrelevant, though it can effect large areas. You can verify that in exposure mode by setting the blackpoint to 15750.

                        However, as several strips of the wedge are affected, which are 1/3 stop apart, the saturation level issue can not be the cause alone.

                        Why don't you upload the raw file, so that I can take a closer look at it?

                        Btw, which ACR version are you using?

                        (Sidenote: I increased the size of the edit boxes, I will send you a message after having uploaded that, probably in a few days.)
                        • 9. Re: ACR Overexposure with Nikon D3
                          Bill_Janes Level 2
                          >Why don't you upload the raw file, so that I can take a closer look at it?

                          Here are two NEF files. The first has blown green channels and the second is the one I used for my post. The green channels are very near clipping. Let me know what you find.

                          http://www.yousendit.com/download/www/azR4eVdvQTZQb0kwTVE9PQ
                          http://www.yousendit.com/download/www/azR4eVd1d0FoMlUwTVE9PQ

                          >Btw, which ACR version are you using?

                          Ver 4.4

                          >(Sidenote: I increased the size of the edit boxes, I will send you a message after having uploaded that, probably in a few days.)

                          Good. Thanks. I am also sending a raw file that Rawanalyze will not read. I think that data was written to it by Nikon Capture NX, which corrupts it for your program.

                          http://www.yousendit.com/download/www/azR4eVdpTk1CTWswTVE9PQ
                          • 10. Re: ACR Overexposure with Nikon D3
                            Panoholic Level 2
                            Bill,

                            I'm afraid I can only join you in frustration.

                            1. Stouff_0003 clips in #3. ACR indicates clipping in #4, but recovery 8 or -1/3 EV is enough to bring #4 back.

                            2. Stouff_0005b: clear from #2, and ACR reports clipping in #2. Again, 1/3 EV.

                            My experience is, that this is a general attitude of ACR, it is not only with the D3 images. I wonder what the designers were thinking when deciding, that they take away 1/3 stop from the dynamic range automatically. Perhaps this is for the uncertainty relating the clipping points of some cameras? 1/3 EV is too much even from that.

                            Re the file ruined by Capture NX: I got used to crooked files, violating the rules of TIFF (for example no image size, or the image content is in MakerNote). Now I have another case for "special treatment": a subfile with zero entries.

                            Solved in the next version.
                            • 11. Re: ACR Overexposure with Nikon D3
                              MadManChan2000 Adobe Employee
                              ACR is not taking away "1/3 stop from the dynamic range automatically."

                              As I noted above, the baseline exposure is simply being set differently.

                              Just set the Exposure in ACR to -0.5 to get the same baseline as NX. Doing so does NOT cause ACR to clip the highlights first and then try to recover it later. The original data is preserved.

                              There is a second discussion point of what the appropriate white level is for the D3. Regardless of what it is, ACR is definitely not clipping 1/3 stop off the highlights ...

                              Cheers,
                              Eric
                              • 12. Re: ACR Overexposure with Nikon D3
                                Panoholic Level 2
                                It is a misconception, that the problem has to do something with Nikon Capture's rendering; it is purely an ACR issue. There is no "baseline exposure" issue here. The situation is very simple: the the top 1/3 stop is transformed into clipping, although ACR knows well, that those pixels have not reached the saturation level. The pixels we are talking about are out of the "misinterpreted" pixel value range.
                                • 13. Re: ACR Overexposure with Nikon D3
                                  Panoholic Level 2
                                  Actually, I do see that ACR assumes a baseline exposure of 0.5, but I don't see why this needs to be so.
                                  • 14. Re: ACR Overexposure with Nikon D3
                                    MadManChan2000 Adobe Employee
                                    Here is an online description of the BaselineExposure tag:

                                    http://www.awaresystems.be/imaging/tiff/tifftags/baselineexposure.html

                                    Essentially it's to accommodate a consistent zero point of exposure compensation across different camera models and different camera vendors (i.e., not just Nikon cameras).
                                    • 15. Re: ACR Overexposure with Nikon D3
                                      Bill_Janes Level 2
                                      >Here is an online description of the BaselineExposure tag:

                                      >http://www.awaresystems.be/imaging/tiff/tifftags/baselineexposure.html

                                      >Essentially it's to accommodate a consistent zero point of exposure compensation across different camera models and different camera vendors (i.e., not just Nikon cameras).

                                      Eric,

                                      As I read the description of the BaselineExposure tag, it is a value written by the CAMERA to the header of the raw file to indicate how much head room is allowed for highlights. The RAW CONVERTER then reads this value and renders the file properly, using the correct amount of exposure compensation. I do not think that it is the job of the raw converter to assign a new value to this tag such that a negative exposure compensation is necessary to render properly exposed images.

                                      Yes, I could merely set -0.5 exposure compensation in ACR with the D3 and get good results, since the raw file is not clipped as it would be in a truly overexposed image. However, this appears a kludge to me that should not be needed. Furthermore, many photographers who had not investigated the behavior of their camera with ACR might assume that the camera is overexposing and reduce exposure accordingly so that no highlight recovery is needed in ACR. In this case, they would not be exposing to the right and would be losing dynamic range and suffer increased shadow noise.
                                      • 16. Re: ACR Overexposure with Nikon D3
                                        Hudechrome-sd9sPI Level 2
                                        My own experience with the D-80 is that the noise at ISO 100 is so low as to be invisible except at really large magnifications. Also, images that open in ACR which appear to have clipping do in fact recover significant highlight information when subject to either exposure comp or Highlight Recovery.

                                        I have been blindsided in the field by using what I call knee-jerk response to clipping showing up in the histograms then adjusting the exposure comp to overcome the difficulty. The next set of images now are wrong!

                                        This is of particular importance when doing a grid array of images which are intended to be stitched into a single image. I expect to set both focus and exposure to manual so that the stitching can proceed most seamless. Therefore, the frame that is most likely to clip has to be determined and exposure set accordingly. Thers is a workaround which requires that the exposure be determined using a wider field of view which encompasses all the values, but this just makes the setup time much longer, and as we all know, nature doesn't wait for anybody.

                                        Using a step wedge here is no different than using one in film, so I guess I'll go have a look.
                                        • 17. Re: ACR Overexposure with Nikon D3
                                          Rob Keijzer Level 1
                                          The second file from Bill Janes (through YouSendIt) is impossible to erase from my Download folder. The system reports it being in use by another program.

                                          It has only the Archive bit set. Same problem after several reboots.

                                          Other files erase fine. No suspect processes in Task Manager.

                                          Does anyone else experience this?

                                          Rob

                                          EDIT: I was able to delete the file in Windows Safe Mode.

                                          Rob
                                          • 18. Re: ACR Overexposure with Nikon D3
                                            Panoholic Level 2
                                            There are legitimate uses of BaselineExposure. For example the Highlight Tone Protection features of newer Canon cameras underexpose by one stop (by reducing the ISO by one stop), and the raw processing is supposed to make up for this adjustment.

                                            However, there is no legitimacy of adding half stop to the Nikon D3 and D300 images. These cameras use the full or almost full numerical range they are recording, and every arbitrary adjustment can cause clipping.

                                            I observed such mutilation by ACR on other raw images as well. For example the Phase One images get a 1 EV boost at certain ISO, 2 EV at another ISO, and lose -1 EV at ISO 100 - this, although the raw data does not justify any adjustment.

                                            The worse on this habit is, that ACR does not give the slightest indication, that the adjustment takes place. I know of these only because I often analyze the DNG format. I just analyzed a "blown out" D300 image, in which no pixels have reached saturation; the photographer did not have any idea that the image gets mutilated by ACR and that it can be "saved".
                                            • 19. Re: ACR Overexposure with Nikon D3
                                              Hudechrome-sd9sPI Level 2
                                              Could you amplify on this, G Sch? How do you open a DNG that would be different than .nef? How do you "save" it? Are you suggesting that converting a suspicious file having highlight clipping might be saved by converting to DNG?
                                              • 20. Re: ACR Overexposure with Nikon D3
                                                Panoholic Level 2
                                                Lawrence,

                                                > How do you open a DNG that would be different than .nef?

                                                The difference between the native raw and the DNG (in this case created by Adobe's DNG converter) is not in the raw data itself but in the metadata; BaselineExposure is a creature of the converter (and if the converter interprets the raw data this way, then ACR too is doing it this way).

                                                I mentioned the DNG format, because *in the DNG file* (as metadata) the adjustment can be seen *explicitely*, in contrast to ACR's processing, which is lying about it: the "Exposure" slider is at 0, despite having adjusted the intensity. (Note, that ACR does not show the adjustment with DNG either.)

                                                > Are you suggesting that converting a suspicious file having highlight clipping might be saved by converting to DNG?

                                                There is no need to convert the file. The data is not lost - it is the same, as if you process a well-exposed image and adjust "Exposure" by (in this case) 1/2 EV: some pixels may become clipped. You can counter this several ways (beside by resetting the exposure adjustment), for example by Recovery (even though there is nothing to recover here, for nothing was lost in the first place), Contrast, etc.

                                                The issue is not, that ACR woud lose image data, but

                                                1. the auto-adjustment is not justified in this case (it is nonsense, that a Nikon D300's exposure should be adjusted to some imaginary "standard", even if that causes loss of part of the image),

                                                2. it can be destructive if the user is not versed enough,

                                                3. it is not indicated explicitely.