29 Replies Latest reply on Aug 10, 2010 7:52 AM by Vit Novak

    JPEG Color Match in ACR

    Noel Carboni Level 7

      Some time ago I set out to create a camera profile for ACR that would do one fairly simple thing:  Match the colors better to those in my camera JPEGs.  It always seemed kind of incredible to me that Camera Raw was not set and could not be set to do this right out of the box.

       

      Having consistent color in all the modes the camera shoots has a lot of subtle advantages - mostly by just not surprising you...  Previews of JPEGs embedded within raw files match what you first see when you open Camera Raw, and in general it just feels better to use (okay, so maybe I have a quirk that way).  That doesn't say you can't change all the sliders the way you like to make the image even better, but at least I have a consistent starting point with the sliders center scale.

       

      The color match may not be utterly perfect (there is only so long you can tweak the profile program before you go nuts), but at least I got it on nearby bases in the same ball field - unlike the results from the canned Adobe profiles.  Here's a screen grab of an emedded JPEG being displayed next to the Camera Raw window on initial settings.

       

      JPEG_ACR_Match.jpg

       

      You might say such a match doesn't matter, and of course you can work without it.  Certainly many people are.  You might even say that the color from the Adobe profiles is more "accurate" or some such.  But I have always quite liked the color right out of my Canon, and this gives me that.

       

      Those of you who look closely may see that I have also dialed in 0.25 EV of negative exposure compensation by default.  This gives me, as a staritng point, a little more overexposure recovery than you get with the default profiles, as I roll off the highlights instead of having them crash into the edge of the graph.

       

      My question is this:

       

      On occasion others have tried my profile, and they claim that it doesn't give them the level of consistency I'm seeing between my own JPEGs and ACR defaults.

       

      Could this be because individual cameras (or lots of cameras) are internally profiled somehow, and thus the mapping of the raw color values to a JPEG image differs from copy to copy?

       

      Have I just not got it "close enough", and other folks are sensing the remaining differences?

       

      Or should I suspect that these other folks have got some of the default settings wrong, or maybe that they're struggling with getting color profiles right on their monitor and system?

       

      These people have never been co-located with me, so I haven't been able to actually see what they're seeing nor check their controls myself.

       

      -Noel

        • 1. Re: JPEG Color Match in ACR
          ssprengel Adobe Community Professional & MVP

          Hopefully the mass-production of sensors and other circuitry is consistent enough so cameras don't need individual profiles from the factory to be similar, but our DSLRs are not laboratory-grade color-measurement devices, that require perfect calibration, either so there is likely some variation.

           

          However, I suspect there is much more variation in monitors and monitor calibrations.

           

          Viewing your side-by-side the ACR version has overly saturated reds, in the skin and especially the dress, which is what you dialed in to ACR's color adjustment, itself.  If this is the closest you can get--41 for the red saturation is too little, then your monitor calibration must be dimming down the reds quite a bit and you don't see the difference.  If you also see your side-by-side as showing an overly red ACR version, then why did you post it as an example of a match?

           

          If you post your raw file as a DNG with the settings already in it, the JPG you're comparing with, and your camera profile, along with the side-by-side screen capture on your system, then people can test if it looks the same on their system as your side-by-side indicates it does on yours.

          • 2. Re: JPEG Color Match in ACR
            Vit Novak Level 3

            Well, I can't say that Camera standard profile for my 400D is especially good match for in-camera jpeg or jpeg from Canon DPP (which also doesn't return the same result as camera with the same setting - for instance, sky is less saturated). Biggest differences are in highlights and in 0.25 less headroom you mentioned. Of course, setting EV to -0.25 or raising Recover to 6 gives this headroom, but colors are also altered at the same time

             

            Is your profile available to download so I can check it with my camera ?

            • 3. Re: JPEG Color Match in ACR
              Noel Carboni Level 7

              Vit, the profile is only good for a Canon EOS-40D.

               

              ssprengel, are you looking at the same image I posted?  The ACR preview is the pane on the right, and if anything the reds are less saturated in that image.  One thing I did with the profile was mute the reds from what Canon delivers a bit.  Bright red colors is an area where the camera overdoes things big time.

               

              Refresh your page, because when I first posted the image I had accidentally left some previously stored parameters active.  I updated it within a few minutes with another capture after having reset all the parameters to defaults.

               

              -Noel

              • 4. Re: JPEG Color Match in ACR
                Vit Novak Level 3

                Noel, I understand that this profile is calibrated for your camera. But I wanted to try it with my 400D, replacing appropriate exif tags inside it (camera model, color matrices) with those for 400D and see the result - if you are willing to share this profile with us, of course

                 

                So as I understand, you made your own program to tweak the lookup table and tone curve inside the profile, or ?

                 

                Thanks

                .

                • 5. Re: JPEG Color Match in ACR
                  JimHess-DIrcbP Level 3

                  You would not be able to to use the profile because it is for a different camera model.  Lightroom will only give you access to profiles for your model.  So it wouldn't do you any good to install that profile unless you know how to modify it and change the model identifier.

                  • 6. Re: JPEG Color Match in ACR
                    Noel Carboni Level 7

                    Problem is, I don't think it can be made to work with another camera.  The Adobe Camera Raw software makes the connection between the camera and which profile to use.  I'm not sure it even makes sense to try to use it with another camera.

                     

                    I used the Adobe Labs DNG PROFILE EDITOR version 1.0 to create the profile (this was a while ago).  Perhaps they've made quite a few improvements to it since then, and I'm anxious to get back into that and see what any new version that's been released can do.

                     

                    I'm willing to share the profile for now, no problem, though I don't guarantee it's "close enough" yet, especially in light of ssprengel's comments above.  I'd prefer not to post it online, though.  Please feel free to eMail me and I'll send you a copy along with instructions about how to set the defaults.  Oh, and I don't use eMail addresses for anything but replying; I hate spam and spammers with a passion.

                     

                    -Noel

                     

                    NCarboni@ProDigitalSoftware.com

                    • 7. Re: JPEG Color Match in ACR
                      Vit Novak Level 3

                      It's in fact very simple to use profile for another camera, but DNG profile editor isn't right tool for that. I used Canon G6 compact camera profile for my A620 (the same sensor, but A620 isn't supported by Adobe, because it has no raw out of the box, but there is CHDK software for Canon compacts that gets you dng files out of the camera). All you need is to change some exif tags in the profile, which itself is a tiff structure (like dng, cr2, nef etc ...). There is utility for this named ExifTool, or another utility dcpTool, which can convert a profile to xml file, so you can edit it with a text editor, and then back to dcp file ...

                      • 8. Re: JPEG Color Match in ACR
                        superchorus

                        Hi, Vit Novak:  I am using CHDK with a Canon A610, and of course ACR does not directly support that model.  I would like to use ACR's profile for another supported model that has the same sensor as the A610. Do you know if A610 has the same sensor as A620, so that I can use the G6 profile like you do? Or do you know of any other camera that uses the sames sensor as A610?  Thanks!

                        • 9. Re: JPEG Color Match in ACR
                          ssprengel Adobe Community Professional & MVP

                          Somehow I missed this reply when looking at the posts in my e-mail, and yes, I was looking at the original copy of the  image you posted, which showed the Calibration sliders, not the Toning  sliders and had saved that copy to my hard-drive, so didn't go back into  the thread on the website to see the new one.  What I see in the  updated one is that the reds on the right are less yellow or more blue, perhaps, but  similarly saturated, now.

                           

                          The thread has moved on, but  can I ask if your own camera-match "profile" includes adjusting the  calibration sliders, too, or just tweaking things in the DNG Profile Editor  and saving to the DCP file?

                          • 10. Re: JPEG Color Match in ACR
                            Vit Novak Level 3

                            A610 has 5 Mpix sensor, while A620 has 7 Mpix. You can try your luck with Adobe profiles for Canon G5 or Olympus C-5060WZ, that are supposed to have the same sensor as A610

                            • 11. Re: JPEG Color Match in ACR
                              Noel Carboni Level 7
                              function(){return A.apply(null,[this].concat($A(arguments)))}

                              ssprengel wrote:

                               

                              The thread has moved on, but  can I ask if your own camera-match "profile" includes adjusting the  calibration sliders, too, or just tweaking things in the DNG Profile Editor  and saving to the DCP file?

                               

                              Minimally, but yes.

                               

                              I discovered that "brightening" the profile curves so that it would produce "proper" results at -0.25 EV exposure compensation (rather than 0 EV) would cause Camera Raw to "fit" more of the image into the final result - effectively making the default retain more dynamic range at the bright end.  Before that, one could plainly see where Canon JPEGs had more highlight detail than Camera Raw was producing.

                               

                              I also changed it so as to set the Brightness slider to center scale, rather than the default +50.

                               

                              I did this all quite a while ago.  What I haven't done is go back to look carefully at the more recent Adobe profiles to see if they may have tidied up any of the issues I originally set out to fix.  When I made this profile up, the out-of-the-box profiles for my 40D produced significant color shifts in a lot of colors.  They may not be so far off any more, and I make no claim to perfection.  My profile brought my results within "close enough" for my own photography.

                               

                              If you want a copy to look the profile over, just eMail me and I'll be happy to send it to you with the specific instructions about what sliders to set.

                               

                              At this point I also still have some images I have been sent to look over, to try to determine whether my profile is still significantly off for some colors.

                               

                              -Noel

                              • 12. Re: JPEG Color Match in ACR
                                superchorus Level 1

                                Thanks.  Actually I tried with the G6 first.  I changed the Exif field

                                "UniqueCameraModel" in that dcp file to "Canon PowerShot A610", renamed the

                                file and copied it in the folder where the other "Adobe Standard" dcp's

                                are.  It worked with my A610 pictures, as on opening them with ACR I could

                                see a new profile in the drop-down list ("Adobe Standard", in addition to

                                the usual "embedded" profile).  But later, I removed that modified dcp file

                                from the folder, did the same using the G5 dcp as a base, and now ACR seems

                                not to recognise the modified dcp (only the "embedded" profile can be

                                selected).  Do you know what could be wrong?  What Exif fields do you edit

                                to make ACR think that the profile is for another camera?

                                • 13. Re: JPEG Color Match in ACR
                                  Noel Carboni Level 7

                                  By the way, I have been testing the newer Adobe profiles, and they are much closer to what the camera delivers than what I set out to do way back when (and Camera Standard seems actually better in some ways at matching color than my profile).  I think I may well start over and make version 2 of this profile.

                                   

                                  -Noel

                                  • 14. Re: JPEG Color Match in ACR
                                    Vit Novak Level 3

                                    I did it with a batch file, which looked like this one (some lines are wrapped here - do a copy-paste to some text editor, which would give you unwrapped lines)

                                     

                                    SET PROFILEDIR=c:\Documents and Settings\All Users\Application Data\Adobe\CameraRaw\CameraProfiles\

                                    SET SOURCEPROFILE=Canon PowerShot G6 Adobe Standard.dcp

                                    SET UNIQUEMODEL=Canon Powershot A620

                                    SET MODEL=Canon A620

                                    SET PROFILENAME=Adobe Standard

                                    copy "%PROFILEDIR%%SOURCEPROFILE%"  "%MODEL% - %PROFILENAME%.dcp"

                                    exiftool "%MODEL% - %PROFILENAME%.dcp"  -overwrite_original  -ProfileName="%PROFILENAME%

                                    exiftool "%MODEL% - %PROFILENAME%.dcp"  -overwrite_original  -UniqueCameraModel="%UNIQUEMODEL%"

                                     

                                     

                                    Be careful with UniqueCamerModel tag (in variable UNIQUEMODEL in this batch) - it must be exactly the same as the same tag in a dng file (watch for spaces etc, although I think it's not case sensitive)

                                    • 15. Re: JPEG Color Match in ACR
                                      Vit Novak Level 3

                                      Noel, recently I was workikng on a similar thing, althoug I did brightening (raising tone curve to compensate for -0.25 EV) with my own program and not with DNG profile editor

                                       

                                      It turned out that simply changing a tone curve gives pretty much the same result as moving recover slider from 0 to 6, where hue and saturation of colors is also changed, so colors are not the same. To retain the same colors at -0.25 exposure setting, there is a need to recalculate whole lookup table in this modified profile, adhering to program code documented in a dng sdk, which is a bit tricky for several reasons

                                       

                                      In one of previous topics, Eric said that he couldn't eliminate this baseline exposure  for a compatibility reasons, but we can hope it whill happen somewhere in the future ...

                                      1 person found this helpful
                                      • 16. Re: JPEG Color Match in ACR
                                        Noel Carboni Level 7

                                        Just based on an hour or so of experimentation and poking around yesterday, it's pretty clear the things that I was working around have all changed, possibly subtly, but I think the answer to my questions at the top of this thread is that what I did was more valid then than now.

                                         

                                        Perhaps I'll do a v2 but from what I can see the Adobe Camera Standard profile with a few tweaks to the color sliders is giving me what I couldn't achieve with them before.

                                         

                                        I do believe a profile that would give an even closer match is possible; ideally I should think Adobe should strive to provide one that matches camera characteristics exactly.  Being able to interchange pretty much freely between camera modes (e.g., JPEG/Raw) is a powerful thing for those who need it.  Given that multiple profiles are possible (simple selection from list), "Camera JPEG Color Match" could just be another entry; it would only add to the utilty, not take away anything.

                                         

                                        -Noel

                                        • 17. Re: JPEG Color Match in ACR
                                          superchorus Level 1

                                          Thank you for your script.  The problem was that I was coying the new

                                          profile into the "...\CameraRaw\CameraProfiles\Adobe Standard" folder.  If I

                                          copy it into "\CameraRaw\CameraProfiles", ACR recognises it.

                                           

                                          By the way, I see from your script that your profiles are located in

                                          "\CameraRaw\CameraProfiles\", unlike mine, which are found in

                                          "...\CameraRaw\CameraProfiles\Adobe Standard" or in folders like

                                          "...\Adobe\CameraRaw\CameraProfiles\Camera\Canon PowerShot G9".  I guess it

                                          must be a different version of Photoshop (mine is CS5).

                                           

                                          Do you find a good match between A620's jpg files and dng files using the G6

                                          profile?  In my case, with the A610 dng files and G5 profile, I find that

                                          they are moderately close to the jpg appearance, but there is much to be

                                          desired (some colours are still off).  Did you have to do some tweaking?

                                          • 18. Re: JPEG Color Match in ACR
                                            Vit Novak Level 3

                                            Hi,

                                             

                                            ACR will read profiles from ...\Adobe\CameraRaw\CameraProfiles\ folder or its subfolders, so I moved profiles I'm using there and left other profiles in subfolders. Adobe standard profile returns different colors than camera jpeg, but I was satisfied with that rendering. However, it was necessary to slightly adjust color matrices. I did it manually, with a short utility I written in Delphi, taking a shot of my test image on a CRT monitor with 400D and A620 and changing matrices in the profile until result from A620 was similaar to that from 400D (which of course is supported by Adobe). It was a while ago, you can check my posts on CHDK forum, my username is vit40. You can adjust color matrices also with DNG profile editor

                                            • 19. Re: JPEG Color Match in ACR
                                              superchorus Level 1

                                              Thanks for your answers.

                                               

                                              I am getting the impression that the problem comes from the fact that the

                                              color matrices (or whatever other parameters applicable) set in the dng

                                              generated by CHDK are not correct for the A610.  When I say not correct, I

                                              mean using the jpg as a reference.  Do you think it would be better to set

                                              CHDK to generate cr2 (instead of dng as I am doing now) and use dng4ps to

                                              convert to dng?  That is, do you think the color matrices used by dng4ps

                                              will be better?  This is really complicated (or I am too new at this)...

                                              what would you do to obtain reasonably close colors in the cr2/dng compared

                                              to the jpg?

                                              • 20. Re: JPEG Color Match in ACR
                                                Vit Novak Level 3

                                                If you select a profile in ACR, color conversion is done using information from the profile (color matrices, lookup table, tone curve), so using dng from a camera or dng generated by dng4ps program won't make any difference. Color matrices from a dng are used only if you select "embedded" profile. Also, I presume that both dng from camera and from dng4ps have the same color matrix, taken from a program dcraw. Color matrices in that program are simply copies of color matrices (D65) from Adobe  - for models that are supported by Adobe. For cameras that are not supported by Adobe, they came from various sources and quality of that calibration is questionable. In CHDK and dng4ps source, color matrices are mostly taken from dcraw, unless someone came withe better calibration (like me for A650 and SX110). In case of my A620, WB was off for more than 1000K and red primary moved way too much towards magenta, so I did my own calibration even before dng profiles were introduced by Adobe. It's not likely that calibration for A610 is much better.

                                                 

                                                Anyway, as I explained on CHDK forum several times and even author of dng4ps didn't want to understand, you can't match colors from jpeg using only color matrix in a dng. In camera, colors are additionally modified to make them look "better", so sky color is saturated, skin colors corrected, midtones brightened etc. So Adobe introduced profiles, with lookup tables and tone curve tables that are used for this. "Camera" color profiles should give the same color rendition as camera (well - similar), while Adobe standard profile should produce similar colors from different cameras and is not intended to emulate jpegs from those cameras

                                                 

                                                There is only Adobe standard profile for G6. So, this one doesn't produce the same colors as camera, but they are not very different (once you calibrate its color matrix, where you can use dng profile editor) - I was quite satisfied with it

                                                • 21. Re: JPEG Color Match in ACR
                                                  superchorus Level 1

                                                  Thanks for your explanation.  Some time ago I tried to use Adobe's

                                                  calibration sliders (hue/saturation for each of the primary colors) to make

                                                  a dng look like the jpeg.  I photographed an image of a color checker, so

                                                  that I could "caliabrate" the dng to look like jpeg.  (My monitor is not

                                                  calibrated, but I believe that does not matter for what I intended to do,

                                                  because even if the colors of the checker on the screen are slightly off,

                                                  they still serve as example colors that I can try to match in dng and jpeg.)

                                                  However, even after arriving at a reasonably close rendition of the colors

                                                  por that picture, in other pictures they were off.  The explanation is

                                                  probably the camera processing that you refer to (saturated sky etc).

                                                   

                                                  If I accept the G5 profile (or some other) as reasonably good for my

                                                  camera, I would like the dng to be "self-contained", so that it does not

                                                  depend on the viewing computer having the (edited and renamed) G5 profile.

                                                  In order to embed the G5 profile in the dng, do you think it would  be

                                                  enough to modify the color matrix (or both matrices)?  Or what other

                                                  information would have to be embedded?  That is, which fields of the dng

                                                  data define the "embedded" profile, besides color matrices?

                                                  • 22. Re: JPEG Color Match in ACR
                                                    sandy_mc Level 3

                                                    The fields that define the embedded profile are the calibration temps, the two color matrixes, the two reverse matrixes, the reduction matrixes, the huesatdelta tables, the look table, and the tone curve. Although not all of those are necessarily present - in early versions of the DNG spec, it was just the two temps and the two color matrixes.

                                                     

                                                    Sandy

                                                    • 23. Re: JPEG Color Match in ACR
                                                      MadManChan2000 Adobe Employee

                                                      Note that for the unsupported models you can use DNG Profile Editor to refine the profile, e.g., calibrate the Temperature & Tint sliders. This can be done in the "Color Matrices" tab, White Balance Calibration. That way the temp/tint numbers will make more sense (e.g., using White Balance = Daylight will give you a reasonable starting point, instead of a strong magenta/green cast).

                                                      • 24. Re: JPEG Color Match in ACR
                                                        superchorus Level 1

                                                        Thans for your comments. How would you calibrate the temp/tint or the six

                                                        sat/hue sliders without having a calibrated monitor?  Is there a technique

                                                        to do it more or less visually? For example, how do you know which settings

                                                        to use for the White Balance Calibration in the "Color Matrices" tab?

                                                        • 25. Re: JPEG Color Match in ACR
                                                          Vit Novak Level 3

                                                          Use Adobe Standard profile for G5 as starting point, as it probably has the same sensor. Take 10-20 typical daylight photos and adjust temperature/tint sliders in DNG profile editor, until ACR shows you temperatures around 5200 K and tint around zero for these photos, when WB is set to As Shot.and when this modified profile is selected. You have to use dngs from camera and not from dng4ps, which don't have As Shot WB info. This will modify Color Matrix1 and Color Matrix 2, which are used only for whitebalancing when forward matrices are present in the profile. When you determine correction for temperature/tint (if needed), continue adjusting other 6 sliders (if needed), trying to match hue and saturation of "memory colors" like skin, sky, foliage etc in jpegs. These sliders move position of primary colors (in Forward matrix 1 and 2), affecting final result (like they do in ACR). Of course, you will never get perfect match raw vs jpeg, but Adobe standard profile is good enough. On the other side, color rendering in camera isn't perfect either. For instance, deep shadows are considerably desaturated, so they have less color noise, but often look quite dull because of it (as shown on some examples on CHDK pages). Fortunately, this is not the case on Canon DSLRs

                                                           

                                                          About embedding a profile in a dng, I never did that and didn't find where to embed the profile (ACR doesn't offer you a possibility to embed a profile in a dng when you save it - must be somewhere else). Obviously, tags that have to be embedded are two calibration illuminats, ColorMatrix1,2, Forwardmatrix1, 2 and Lookup table. Tone curve is not present in Adobe standard profiles, as they use default tone curve embeded in ACR (however, when you save a modified profile with DNG profile editor, it will contain this curve). In case of dng from camera, you actually also have embeded profile, but it consists only of one color matrix

                                                          • 26. Re: JPEG Color Match in ACR
                                                            superchorus Level 1

                                                            Thanks... I'll proceed that way.  Anyway, I am getting the impression that

                                                            jpg's from the camera are not necessarily something to aim at, with all

                                                            those tricks of saturating the sky, desaturating shadows etc.

                                                             

                                                            I am also surprised at how difficult it is to properly remove noise from dng

                                                            files... The camera jpeg always looks less noisy, and at the same time

                                                            sharper! I wonder what kind of tricks the camera uses here...  In-computer

                                                            noise reduction should be better than in-camera, because better algorithms

                                                            can be used in the computer, but that is definitely not the case, at least

                                                            with ACR's noise reduction (and even if ACR in PS CS5 seems to reduce noise

                                                            better than earlier versions).  Somehow the camera knows how to handle its

                                                            own noise better...

                                                            • 27. Re: JPEG Color Match in ACR
                                                              Vit Novak Level 3

                                                              After saving new profile with profile editor, check also the size of the file, should be similar. When I started with A620 profile, I used 400D profile as template and after saving, it lost lookup table, resulting in much smaller file (can't remember details why)

                                                               

                                                              About in-camera processing, yes, NR is quite efficient, but much of the details are also lost. I'm not a fan of Canon's algorithms (whether in camera or DPP), they seems to stick to the same old gradient interpolation method for ages (unless something changed in very last models / vertsions of DPP). In my opinion, interpolation in last versions of ACR is much better, sharpening doesn't produce annoying 'digital look' and NR is also improved. Anyway, color processing in ACR is still behind camera/DPP when using camera profiles for 400D - these profiles are not very accurate and dealing with highlights / out of gamut values need improvement as I demonstrated with two samples in another topic

                                                              • 28. Re: JPEG Color Match in ACR
                                                                MadManChan2000 Adobe Employee

                                                                Hi Vit, to embed a color profile in DNG, you can first select the profile you want to use in the Camera Calibration's "Profile" menu, then choose "Update

                                                                DNG Previews" from the flyout menu (little triangle on the right side). Effectively this forces the headers, previews, profile, etc. to be rewritten, and also causes your selected color profile to be embedded.

                                                                • 29. Re: JPEG Color Match in ACR
                                                                  Vit Novak Level 3

                                                                  Thanks, Eric.