15 Replies Latest reply: Oct 6, 2011 10:59 PM by Vit Novak RSS

    How Camera Raw and Various Profiles Handle Transition Into Overexposure

    Noel Carboni Community Member

      With all the attention I've been paying to profiles lately, I have been opening a lot of images with bright blue colors (hence my recent post on trying to get better blue sky color). 

       

      One of the things I've noticed is that the transition into overexposure appears to be handled VERY differently depending on the profile chosen.

       

      Of particular interest is an image I shot recently toward a stage where some bright blue lights were shining on the crowd and into my Canon EOS-40D's lens.  I was surprised at how differently the transition into overexposure is made depending on the profile chosen.

       

      Notably not one of the Adobe-supplied profiles handles this transition smoothly at all, showing some sharp transitions - which helps explain why I had been having such trouble with sky color.

       

      Some examples follow (these have all been converted to sRGB for display via browser).  Specifically note the area around the bright blue light in the upper-right.

       

      Adobe Standard profile, as supplied with Camera Raw by Adobe:

      AdobeStandardProfileBlueOverexposure.jpg

       

      Camera Standard profile, as supplied with Camera Raw by Adobe:

      CameraStandardProfileBlueOverexposure.jpg

       

      Camera Landscape profile, as supplied with Camera Raw by Adobe:

      CameraLandscapeProfileBlueOverexposure.jpg

       

       

      The very good 40D "DPP Standard sRGB" profile Vit Novak provided me a couple of days ago, based on his measurements of how Canon DPP handles color:

      VitNovak40DProfileBlueOverexposure.jpg

       

       

      The in-camera JPEG:

      InCameraJPEGBlueOverexposure.jpg

       

      -Noel

        • 1. Re: How Camera Raw and Various Profiles Handle Transition Into Overexposure
          Vit Novak Community Member

          Noel, can you post the cr2

           

          I want to try a code to smoothen up the highlights. I've been testing it on blown sky, but I don't have a raw like this, as 99% of my photos are daylight. I can make a transition smoother, although with existing ACR workflow I think it's not possible to make the same degree of matching with in-camera jpeg as in non-blown parts of the picture.

           

          Older Adobe Camera profiles perform badly in this area because of several reasons - baseline exposure tag (causing unnecessary clipping), matrix color space transformation and clipping into Photo Pro (especially in blue color, as blue virtual primary position is outside Photo Pro gamut for most cameras) etc ... Things get worse if you use exposure slider and many times even recovery slider, because profile is calculated on clipped data ... etc ...

          • 2. Re: How Camera Raw and Various Profiles Handle Transition Into Overexposure
            Noel Carboni Community Member

            Vit Novak wrote:


            Noel, can you post the cr2

             

            I did, though the link isn't obvious.  Click on the text "an image I shot" in the original post.

             

            To be honest, your profile already works so well I wasn't thinking it needed to improve, though I've noted (as you mentioned) that if I don't keep a bit of negative exposure compensation dialed in the transition gets more abrupt.

             

            -Noel

            • 3. Re: How Camera Raw and Various Profiles Handle Transition Into Overexposure
              Vit Novak Community Member

              Thanks.

               

              Actually, at default exposure setting (-0.24), my profile also produces abrupt banding around the strongest blue light. However, it's gone by decreasing exposure to about -0.4, or decreasing saturation by 5-6%, or applying recovery 4-5. This banding already exist on the raw - it's where the green channel gets clipped. It looks like the camera / DPP has a sort of automatic recovery procedure or some other mechanism to deal with this, because this part of the picture is rendered differently than the same colors on the test image, so I think I'm not able to emulate this with a dng profile, although I can probably reduce banding at this position in the profile

               

              However, while camera deals better with banding than my profile or Camera Standard , hue is even more wrong - it drops to around 160 (in sRGB), while the original hue of the light is 220 or more. Restoring highlights is always guessing, as information is missing

               

              I hope somewhere in the future we'll get sensors with better dynamic range ...

              • 4. Re: How Camera Raw and Various Profiles Handle Transition Into Overexposure
                Noel Carboni Community Member

                Yes, I have my default now at -0.5 EV plus a small boost (+62) of Brightness, so the highlights roll off better yet the overall tonality is much the same.  Is it possible to scale the numbers in your profile so that negative exposure compensation is not needed?

                 

                To be fair all one needs to do to avoid overexposure is underexpose a bit and boom, there's more headroom.  In my opinion the headroom provided by my camera already per the metering and histogram on the body itself is fine for virtually all uses.  My whole point with this thread is that how the system performs at the edge of the envelope can vary depending on choices we make, and which are not always obvious.

                 

                And yes, I noticed that with your DPP standard sRGB profile the blue stays a better hue than with the camera JPEGs.  That's all the better for bright skies.  I am certainly not upset that the Camera Raw converter can be coerced into outperforming the camera's own conversion - it just doesn't do so out of the box.

                 

                -Noel

                • 5. Re: How Camera Raw and Various Profiles Handle Transition Into Overexposure
                  Vit Novak Community Member

                  Here is another version of Standard profile, where I did some interpolations in highlight range. It behaves better with above picture at default setting of exposure (-0.24) and above. Also, I corrected strange hue shift of near-blown sky present in jpegs from camera. However, it's not possible to eliminate banding around the light present in raw and there will still be some photos rendered better by the camera - can't do better with my current knowledge

                   

                  It's not possible to eliminate negative exposure correction until Adobe implements this option in dng standard and ACR, unless I do the same as Adobe in profiles for 40D, which results in highlight clipping and issues connected with that

                   

                  http://www.megaupload.com/?d=2NYD4UWD

                  • 6. Re: How Camera Raw and Various Profiles Handle Transition Into Overexposure
                    Noel Carboni Community Member

                    That's awesome!

                     

                    For "normal" exposure levels it exactly matches the other images converted with the profile you developed before, but the hard edge (e.g., seen when the edges of the image are lightened by Vignetting Correction) is gone.  Well done!

                     

                    I tried it on a number of other images and found that the transition from blue sky to whited-out overexposure was indeed noticeably better, with the sky going to cyan slightly less quicky.  For example, I have an evening time exposure with the sky transitioning to overexposure.  I converted the image once with Adobe Standard and each of your profiles, and put an extreme Curves adjustment layer over the top to more easily see the transition.  Note the color, in particular, of the sky transition into overexposure with the three different conversions, your most recent on the bottom.

                     

                    SkyTransitionThreeProfiles.jpg

                     

                    You really have gotten your head around how these profiles work with Camera raw, and you do damned good work with that knowledge, Vit!

                     

                    Let's review the features of this profile:

                     

                    1.  It matches the camera color virtually perfectly, which I for one find an excellent starting point for further Camera Raw adjustments.

                     

                    2.  It handles transition into overexposure IMO as well or better than the camera manufacturer's software and hardware.

                     

                    3.  Since you developed it to match the camera manufacturer's sRGB conversions, and THEY don't think channel clipping is acceptable if the user has chosen to convert to sRGB, it has solved the problem with channel clipping that I had seen with all the Adobe profiles when converting directly to sRGB, and has thus made the action I had been using to do a "bend the edges" conversion unnecessary.

                     

                    There is nothing not to love here!

                     

                    Vit, I don't know how much work is involved with making such a profile - clearly you've put some serious time into getting where you are, but would you be interested in partnering with me and selling sets of profiles with characteristics like this one to others who would like to get the best possible Camera Raw or Lightroom conversions?  If so, let's talk off the forum via eMail.

                     

                    -Noel

                    • 7. Re: How Camera Raw and Various Profiles Handle Transition Into Overexposure
                      Noel Carboni Community Member

                      Here's the kind of image where it makes a real difference:

                       

                      LandscapeSky.jpg

                       

                      -Noel

                      • 8. Re: How Camera Raw and Various Profiles Handle Transition Into Overexposure
                        Vit Novak Community Member

                        Ok Noel, I'm glad this worked fine for you

                         

                        Several months ago, in one post I stated that I wouldn't underestimate knowledge of companies like Nikon and Canon. Jeff didn't quite agree with me about that, claiming that Adobe was much longer in color business. However, 5 years (or so) ago, when I bought my 400D, it had quite good color management and various things like dealing with highlight clipping were sorted quite good. At that time, PS didn't have profiles at all, and in some aspects even newest profiles don't do the job as good

                         

                        I made this mostly out of curiosity and because my Canon compacts were lacking ACR support. And to stimulate Adobe to improve color management. Of course, it was a big waste of time. I have no intention to sell these things - I didn't invent anything new, I just tried to extract color profiles from the camera / DPP so they work almost as well as in camera

                         

                        Also, it would be good if somewhere in the future Adobe provides better profiles for older cameras

                        • 9. Re: How Camera Raw and Various Profiles Handle Transition Into Overexposure
                          Yammer Community Member

                          I haven't been following this too closely, but isn't the way highlights are badly rendered partly to do with baseline exposure in ACR? Looking at the example above, I'd guess an exposure offset of half a stop would fix the strange blue hue and banding. My own defaults include -0.5EV exposure compensation, and, as I try to expose to the right, this means my high DR shots are often underexposed intentionally to preserve highlights. This is usually compensated by a mixture of small amounts of highlight Recovery, a boost to Brightness, and a suitable amount of Fill, un-flattened by a moderate amount of Clarity.

                           

                          One thing I don't do is adjust Exposure for the midtones, which is how I'd expect it to work, as this tends to provide washed-out highlights in ACR.

                          • 10. Re: How Camera Raw and Various Profiles Handle Transition Into Overexposure
                            Noel Carboni Community Member

                            Thing is, depending on the subject material, you might have a perfectly well-exposed image overall but have a few blown (or nearly blown) highlights - the banner image at the top of this thread is a good example.  Sure, you can expose for the highlights, and that will bring your main subject way down in brightness so that when you do the processing it brings out more noise.

                             

                            But why bother?  My point here is that the data is already in there to make better images than Adobe's profiles will facilitate (and even, as it turns out, better than that delivered by the camera maker's own software/firmware).  Adobe may feel they know more about color than everyone else, but a demonstrated complete ignorance of how to handle data at the edges of the camera's capabilities says otherwise!  Vit clearly understands this.

                             

                            A year or two ago I created a thread here where I asked why, when I converted raw files directly to sRGB, that I would see channel clipping.  I was told that this was because the camera was capturing colors outside of the sRGB gamut, that such clipping was a fact of life, that some new profile format would be needed to fix the problem, and that I was all but stupid for asking such a thing because clearly I didn't understand color management - EVEN THOUGH Canon had long since dealt with the problem and would produce beautiful sRGB output sans clipping from the very same raw files! 

                             

                            Now, lo and behold, here we see a Camera Raw profile that works with that exact same Adobe software that does the job beautifully.

                             

                            But hey, I'm happier than a clam now...  I now make better images from my 40D than everyone in the world who hasn't downloaded Vit's profile. 

                             

                            -Noel

                            • 11. Re: How Camera Raw and Various Profiles Handle Transition Into Overexposure
                              Yammer Community Member

                              I get what you're saying now.

                               

                              It's similar to an epiphany I had a few months ago, when dealing with a small number of photos with very wide gamut colours.

                               

                              I always (wrongly) assumed that "rendering intent" was very clever in the way that images were converted from one colour space to another. I understand the semantics of the different intents (e.g. gamut clipping as opposed to gamut compression), but in practice this didn't match reality.

                               

                              Some may remember my incredibly red tulips being converted from ProPhoto to AdobeRGB to sRGB. I understood that conversion of the image to a smaller space using relative colormetric intent would lose some of extreme reds and turn sections of the image to solid red, but what confused me was how perceptual intent produced equally useless conversions. I'd expected a hue shift in order to accomodate the extreme reds, but the results were more or less the same as before. It was explained to me that I would have to manipulate HSL before attempting any colour space conversion. So, I still ask myself what the point of perceptual conversion is.

                               

                              You're quite right in expecting ACR to deal with highlight areas more satisfactorily. My suggestion was in effect a workaround, which shouldn't be required. Of course you should adjust for midtones, then deal with highlights and shadows later. And you shouldn't have to come up with workarounds to return proper saturation to skies and skin, just because they are very bright.

                               

                              Like Vit said earlier, it might be cool to see some better camera profiles. I'm not sure how that would help me though, as I generate my own using Adobe's profile editor software and a ColorChecker. Maybe I need to buy Vit a beer and get him to make me one too. What's your poison, Vit?

                              • 12. Re: How Camera Raw and Various Profiles Handle Transition Into Overexposure
                                Noel Carboni Community Member

                                Just another good example of where improved handling by Vit's profile of the transition into very bright exposure levels made a very visible difference in an end result...

                                 

                                AnotherExample.jpg

                                 

                                -Noel

                                • 13. Re: How Camera Raw and Various Profiles Handle Transition Into Overexposure
                                  Vit Novak Community Member

                                  About rendering intents ... icc profiles also evolved during timeline. sRGB.icc on my computer is only 3 kB in size. It's a matrix profile and no matter what rendering intent I select in PS, result is the same (can't be different I suppose). However, on icc site, you can download sRGB v4 profile, which is about 60 kB in size and has matrix for colorimetric rendering and lookup table for perceptual rendering, which means compression of bigger color space into sRGB and various adjustments of hue/saturation/brightness.

                                   

                                  Rendering intent in ACR is colorimetric - out of gamut colors are clipped by clamping R, G and B into range 0..1

                                   

                                  Rendering intent in Canon cameras and DPP is a kind of perceptual - out of gamut colors are compressed into sRGB or AdobeRGB (at least partially - on top of the gamut it looks like they are also clipped, hence cyan tone of near-blown sky in sRGB mode)

                                  • 14. Re: How Camera Raw and Various Profiles Handle Transition Into Overexposure
                                    Noel Carboni Community Member

                                    Canon didn't forget that making pretty pictures is part of the goal, even if that involves sacrificing mathematical accuracy. 

                                     

                                    The cyan shift may be intentional - to help the luminance seem to move up to pure white more smoothly.  Note that your latest profile does have a bit more of an abrupt jump to the desaturated bright space just around that bright blue spotlight in my test image above.  Your first profile seemed to make the transition a bit more smoothly (similar to the in-camera processing), but then it had the characteristic that any positive exposure compensation (e.g., vignetting compensation) would push it into some kind of overflow, revealing a hard ring.

                                     

                                    -Noel

                                    • 15. Re: How Camera Raw and Various Profiles Handle Transition Into Overexposure
                                      Vit Novak Community Member

                                      This hue shift in case of blue/cyan is intentional, also because blown blue colors are rendered around sRGB hue 190 deg, so transition has to be smooth. But in case of sRGB profile, it is abrupt somewhere around half of LUT (depending on saturation), where sRGB nonlinear values reach 1, while in Adobe RGB it's much smoother. It looks like it is clipped into sRGB

                                       

                                      Problem about highlights and possible occurence of hard ring is that Canon seems to has built-in LUT that extends above brightness value 1, to cover sensor output after whitebalancing, where R and B can be above 1 (because R and B channels are less sensitive and they are multiplied with WB R and B multipliers). Dng profiles don't offer this possibility - everything above 1 is clipped before LUT calculation, so I adjusted calibrating procedure to this difference, but it can't be handled properly. So whatever I do, it won't be the same in this region (+ there can be some additional logic in the camera). But ACR has recovery slider, which compresses upper region to fit into LUT

                                       

                                      This subject is quite complex, really