21 Replies Latest reply on May 10, 2013 8:00 AM by MadManChan2000

    Homemade camera profiles

    Yammer Level 4

      As an early adopter of DNG Profile Editor and user of the Chart Wizard for creating my D300 profiles for Camera Raw, I have used each successive version as it has been released. It worked quite well with my D300, and was a definite improvement over the profiles supplied with ACR, although I had to give it a bit of help with the values, by taking the mode of a few saturation readings and adding a bit of manual tweak to lightness.

       

      I acquired a D800 in March, and headed outside on the first sunny day with the GM ColorChecker to take a few shots. I used the latest version of DNGPE to generate a camera profile, which seemed to be an improvement over Adobe Standard in terms of colour accuracy and balance. However, a couple of weeks later, I noticed some unpleasant banding on the boundaries between certain colours. I decided to sacrifice accuracy for aesthetics, and ditched the profile.

       

      I guess that Chart Wizard isn't really accurate enough for my needs, or the patch transforms are so few and so extreme that it can't provide a smooth transition between some colours. Am I doing it wrong, or is this a known limitation?

       

      Are there any plans for a new version of DNG PE? Is this the sort of question no one is allowed to answer?

        • 1. Re: Homemade camera profiles
          MadManChan2000 Adobe Employee

          We are continuing to work on internal camera profile-building methods, some of which may find their ways into an updated version of DNG PE in the future (but no timetable for that at present).

           

          In the short term, the issue you're likely running into is a couple of the color control points are too close together, which sacrifices some smoothness in exchange for the degree of hue/saturation control that can be achieved.  A solution to that is to move (manually) the control points a bit farther apart, and also to make nearby control points have less-strong adjustments.  In other words, if you have two control points that are near each other and they result in dramatically different adjustments (one yanks the colors one way, and the other yanks the colors another way), that will cause "stretching" problems in the areas around those colors.

          • 2. Re: Homemade camera profiles
            Yammer Level 4

            Thanks, Eric. I assume that the colour points created by the Chart Wizard always start off in the same place for any camera. So when you say, "a couple of the color control points are too close together," do you mean that they end up too close together?

            Maybe I should persist with the profile a bit longer. I do prefer the colours I get with the Chart Wizard, although I'm not keen on the side effects. I did notice that the latest version of DNG PE produces stronger saturation shifts on some patches (especially Skin) with my last camera. Could you say what were the main differences between the last two versions (1.0.0.45 & 1.0.0.46)?

            • 3. Re: Homemade camera profiles
              Yammer Level 4

              P.S., here is a screen grab of the side effect (click to enlarge) and the corresponding Color Table.

              _KN50813DPE.jpg

              DNGPE.jpg

              • 4. Re: Homemade camera profiles
                Vit Novak Level 3

                1st and 7th pointh are most likely to be the reason for that problem. They have similar hue and saturation, but their saturation correction is considerably different. 1st point significantly desaturates the red-orange color, making it much lighert, while 7th does not. So besides banding, it would also greatly increase color noise (visible with compacts or high ISOs)

                 

                Making corrections in HSV isn't really optimal way to do these corrections anyway, but using some other approach would also have some drawbacks. For instance, using Lab will not result with this big differences in lightness, but is known to produce perceptual hue changes for some colors etc. But main question is - why correction for two very similar colors is that different

                • 5. Re: Homemade camera profiles
                  Yammer Level 4

                  Vit Novak wrote:

                   

                  1st and 7th pointh are most likely to be the reason for that problem. They have similar hue and saturation, but their saturation correction is considerably different. 1st point significantly desaturates the red-orange color, making it much lighert, while 7th does not. So besides banding, it would also greatly increase color noise (visible with compacts or high ISOs)

                   

                  Making corrections in HSV isn't really optimal way to do these corrections anyway, but using some other approach would also have some drawbacks. For instance, using Lab will not result with this big differences in lightness, but is known to produce perceptual hue changes for some colors etc. But main question is - why correction for two very similar colors is that different

                  The patches are automatically generated by the DNG PE Chart Wizard, based on the ColorChecker board. Although it doesn't look like it in the table, the first patch is 'Dark Skin' (brown) and the seventh is 'Orange'. They don't look that similar in real life.

                   

                  Just checking: the problem area for me is on the frayed edge of the flag, where orange meets sky. The image will have a gradient between orange and blue where there is blur. This to me is where the banding is the most unpleasant, and looks almost over-sharpened, with a light halo on the flag and a dark halo on the sky. The sharpening settings are normal, and the Adobe profiles do not produce this effect, so it's definitely down to the homemade profile.

                  • 6. Re: Homemade camera profiles
                    Yammer Level 4

                    ...By the way, since my earlier post, I made another profile based on the one above (made with DNG PE 1.0.0.46), but with saturation values reduced to 25%, and the effect was mostly gone. I also made a fresh profile using DNG PE 1.0.0.45 which had no side-effects.

                    • 7. Re: Homemade camera profiles
                      Vit Novak Level 3

                      Yes, I see now (didn't check the photo before), blue-violet color on the edge (aprox. hue 240-330) is looking too desaturated

                      • 8. Re: Homemade camera profiles
                        ssprengel Adobe Community Professional & MVP

                        If you haven’t already tried, you might start with a different base profile besides Adobe Standard, one of the Camera-match profiles, and see if your generated profile has less severe adjustments.

                         

                         

                         

                        Are you saying profiles for your D800 made with DNG PE 1.0.0.45 don’t have the problem that the DNG PE 1.0.0.46 profiles had when both started with Adobe Standard?   If so, then what color-patches had significantly different values between the two DNGPE versions?

                        • 9. Re: Homemade camera profiles
                          Yammer Level 4

                          The hue shifts are very close, it's the saturation shifts which vary a lot, mostly 2 (Light Skin), 5 (Blue Flower) and 6 (Bluish Green), where the changes are in double figures. I've added a table below, based on running the same DNG through the Chart Wizard on DNG PE 1.0.0.45 and 1.0.0.46.

                           

                          Actually, the more I look at the figures, the more I'm inclined to go back to version 1.0.0.45, where the highest saturation shift (by a long way) is 10.

                           

                          PE.png

                          I ought to shoot some more ColorChecker tests, but the weather isn't ideal today.

                          • 10. Re: Homemade camera profiles
                            Vit Novak Level 3

                            Yes, I would strongly suggest keeping these corrections low in order to avoid problems. Adobe profiles sometimes don't perform very well even out of the box. Check result of Adobe profile for Panasonic compact camera LX5 (out of the box, without any corrections) vs one of my experimental profiles for LX5 (which still isn't as good noise-wise as recalibrated Canon profile). PV2012, NR was turned off in both cases to show the difference in noise and medium amount of sharpening was used. It's not that easy to get optimal results  ... so I hope Adobe will try to improve these things overall somewhere in the future

                             

                            SkySample.jpg

                            • 11. Re: Homemade camera profiles
                              deejjjaaaa Level 2

                              > I acquired a D800 in March, and headed outside on the first sunny day with the GM ColorChecker to take a few shots. I used the latest version of DNGPE to generate a camera profile, which seemed to be an improvement over Adobe Standard in terms of colour accuracy and balance

                               


                              says Eric "...It's unlikely that a custom profile will help you with one flavor of daylight more than another.  This is because natural daylight has a pretty full and reasonably smooth spectrum whose changes can be well approximated by per-channel scaling (i.e., simple white balance).  You'll see the benefit of additional custom profiles once you start working with light sources that are spectrally quite different (e.g., compact fluorescents)..."

                               

                              so I guess you can't get matrix part better than they supply in their standard profiles (which are I 'd assume not done by shooting targets but rather using monochromators) - you can only get alternative LUT tables more suitable for your taste... but that's about taste and not about accuracy... and you may be better off by just editing LUTs in supplied profiles instead of wasting time shooting colorcheckers under some daylight.

                              • 12. Re: Homemade camera profiles
                                Yammer Level 4

                                If there was a better way to do it, I would. I'm not shooting ColorCheckers for fun, I do it because the colours in the canned profiles aren't quite right to my eye. I don't don't know if that's taste or accuracy, but, for example, skin always looks a bit red/ruddy, and it does look better when I use the ChartWizard. So how does DNG PE know what my taste is for skin tone? I always saw the process as fine tuning manufacturing tolerance.

                                • 13. Re: Homemade camera profiles
                                  Vit Novak Level 3

                                  Color accuracy and taste for some color are different things. Adobe standard profiles are meant to be accurate, according to what they say. However, this accuracy in case of my two cameras that are supported by ACR (400D and LX5)  is relatively poor on ordinary photos, especially in blue range. I experimented much with these things in the past, not only with back-engineering camera profiles, but also tried to make my own profiles. Main problem that I see in default ACR workflow is implementing a tone curve in Linear photo pro color space. Primaries of that color space are quite different than of sRGB and AdobeRGB, so there is a number of issues. Noise issue illustrated above is one of them. It seems that other hardware and software manufacturers are implementing tone curve in output color space, bacause when I made profiles that are simulating that workflow, I actually got quite similar results like canon and panasonic camera profiles, more acccurate overall and with less issues. This also seems to be the reason why color matrices calculated by Adobe are significantily different than matrices for the same cameras calculated by some other manufacturers etc ...

                                   

                                  So in my personal opinion, DNG PE is of no much use - it won't correct mentioned issues and will only introduce new issues by trying to correct some colors ... My personal opinion only, so take it or leave it

                                   

                                  Taste for color is different thing. That's why there are various profiles in camera, where colors are tweaked to get (over)saturated sky and nice looking skin for instance. It's not that much about accuracy, but about pleasant looking photo

                                  • 14. Re: Homemade camera profiles
                                    Yammer Level 4

                                    Vit, I fully accept what you say. I appreciate that what's accurate and what's "good" are two different things, but...

                                     

                                    Profiles are created by Adobe using real cameras, but not my cameras. I always guessed, rightly or wrongly, that there are subtle differences between cameras of the same model, maybe due to batch differences in Bayer filters, or calibration conditions. I can't think of another reason which would explain why, whenever I've used DNG Profile Editor on one of the five raw-shooting cameras I have owned, the "corrected" colours look more realistic than the default. I always thought that was the main point of DNG Profile Editor; and not for creating profiles for odd lighting or special effects.

                                    • 15. Re: Homemade camera profiles
                                      Vit Novak Level 3

                                      Yes, of course they looked more realistic because you corrected some of colors and most of them are memory colors. However, as you noted, this introduces other issues

                                      Surely there are also differences among cameras and lenses. Hard to say how big they are without testing, but I don't believe that differences among sensors from different batch are very big. There are more differences regarding different lenses, but however, my LX5 compact doesn't have interchangeable lens and color accuracy of Adobe standard is still poor, much worse than built-in camera profile

                                      • 16. Re: Homemade camera profiles
                                        MadManChan2000 Adobe Employee

                                        The default color points chosen by the DNG PE Chart Wizard are not necessarily the same for all cameras.  They are chosen based on (1) the Base profile, and (2) the actual raw values recorded in the image being used for the wizard.  So for some cameras the control points will be naturally spaced reasonably far apart (and those that are near may have similar corrections), whereas for others some control points may be too close to each other (and in some cases the nearby ones will have differing corrections).

                                         

                                        As to why nearby control points may have (greatly) differing corrections, the two common culprits are (1) uneven lighting, e.g., a color cast on the image itself, or (2) camera metamerism.  #1 can be fixed by changing your lighting conditions and is pretty straightforward, esp. with a small chart, but there's not much you can do about #2 (this depends on the color filters chosen by the vendor).

                                        • 17. Re: Homemade camera profiles
                                          Yammer Level 4

                                          I have always used Adobe Standard as the base profile. It's either that or the emulation profiles, so it seemed obvious to go with the Adobe one.

                                           

                                          I don't understand what you mean when you say "The default color points chosen by the DNG PE Chart Wizard are not necessarily the same for all cameras". What do you mean by "color point"?

                                           

                                          I have always used outdoor sunlight, which I would have thought was pretty even. Would blue sky be classed as a colour cast? Am I best pointing the chart at cloud?

                                          • 18. Re: Homemade camera profiles
                                            Vit Novak Level 3

                                            I actually never really used DNG PE, but out of curiosity, tried making a calibrated profile for D800, based on Adobe standard and using a test photo that I got from one site with Macbeth on it, just to see how it works. Tried DNG PE .45 and .46 and results are indeed different - similar to what Yammer got with his test photos.

                                             

                                            After examining the recipe files, my understanding is that for all patches in the chart there is a hue and saturation after applying base profile (SrcHue, SrcSat) in linear Digital photo pro. However, while hue values were similar with both versions, saturation was different - in case of .45 usually somewhat lower than on photo converted with ACR and in case of .46 higher. Which is a bit strange to me. These values are compared to correct values to calculate LUT for making a new, calibrated profile. So, calculated corrections for these colors (SatAdjust) are different for .45 and .46. One explanation could be an additional tone curve in the profile generated  with version .46, while profile generated with .45 has no tone curve,  meaning standard Adobe curve is used by ACR.

                                             

                                            I understand that these values (SrcHue, SrcSat) are different for every camera, because in some cases Adobe standard profile does better and in some worse job (depending on the sensor and lens) + different lighting conditions, so for some cameras, two points can fall closer and for some cameras they won't be that close. However, it would be nice if algorithm for calculation of lookup tables can take care not to make things too abrupt, because except correcting the lighting, there is nothing user can do to obtain better result, except of course making manual corrections ... Also, I don't see any nonzero ValAdjust value in recipe files, and without correcting Value while making Sat and Hue corrections, it's very unlikely that resulting profile will perform well - there will be increased color noise / banding - at least it was always the case when I was playing with my own algorithms to do similar things ...

                                            • 19. Re: Homemade camera profiles
                                              Yammer Level 4

                                              Thanks for your efforts on this, Vit. I had to read your post twice to take it all in.

                                               

                                              I'm glad you saw the same differences as me between the latest two versions of DNG PE. I was starting to think I had done a bad job of shooting the ColorChecker. .46 maxed out the saturation offset of one patch on one of my shots; I'm starting to think that this behaviour can't be right, especially with a camera like the D800.

                                               

                                              Maybe I'll experiment with DCPtool a bit, now I've got a copy. I don't understand the internals of DCP at all, but it sounds like adjusting the source is the best way to get good results, so it might be worth going through the learning curve.

                                              • 20. Re: Homemade camera profiles
                                                Vit Novak Level 3

                                                However, it doesn't need to be an indicator of an error - obviously algorithms used in DNG PE changed and it looks like values in recipe files have different meanings. After trying resulting profiles, I see that tone curve is the same (or almost) in both versions and most colors look very similar - except the blue-violet range which is less saturated in .46 version. Looks like a step into right direction to me, because biggest color error of Adobe standard profiles for my cameras was here. But - I have a feeling that more steps are needed to make it right / accurate ...

                                                 

                                                Made a typo in previous post, of course it was Photo pro color space and not Digital photo pro ...

                                                • 21. Re: Homemade camera profiles
                                                  MadManChan2000 Adobe Employee

                                                  Yes, the limitation of the chart wizard in DNG PE at present is that it tries to map color patches exactly, and that's what leads to smoothness problems if two patches are too close together in HS space.  The reason for this is that DNG PE was originally designed as a manual color editor, where the user could "lock down" a specific color by clicking on a color in an image, and also freely adjust other colors, again by clicking on those colors and then adjusting them.  So, that type of precision was (and remains) very useful for that type of direct editing/tweaking. 

                                                   

                                                  For the case of auto-building profiles using some type of chart as guidance, it would be better if DNG PE had some type of smoothing or relaxation in its table generation to avoid some of the problems discussed here.  But the current public version of DNG PE does not have that.  Perhaps something for the future.