35 Replies Latest reply on Apr 11, 2010 4:06 AM by tereza29a

    D700 Camera Profile Artifacts (still present in ACR 5.5?)

    photo81

      Just installed ACR 5.5 RC and it looks like the color problems with the "Camera" profiles (for the Nikon D700 and D3) are still present. Namely, green/magenta color casts in what should be neutral areas, and weird posterization effects in shadow areas. Then again, I'm not sure if  installing a new version of ACR update camera color profiles.

       

      I posted a thread about this on Dpreview a while ago; or just do a Google search for "D700 color" and it's one of the top results.

       

      Here's a Flickr photoset with some crops showing the issue. The artifacts with the shadows are kind of hard to see in those small crops though (it's better to look at the original image).

       

      To Adobe: can we get any sort of an idea when this is going to fixed?

        • 2. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
          photo81 Level 1

          I wonder how long it takes to "investigate" and fix something like this. It could be a long time until the next release of ACR or Lightroom.

           

          I'm pretty sure there a lot of D700/D3 users out there using Lightroom/ACR, so I'm hoping that this is more of a priority for the Camera Raw team. The existing profiles are pretty much unusable right now, and I'm holding off editing a lot of pictures until a fix comes out (hopefully sooner than later, which would be much appreciated).

          • 3. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
            sandy_mc Level 3

            What you're seeing - the color shifts anyway - are as a result of the "hue twists" in the new generation Adobe profiles. What it amounts to is that the new profiles change the hue of a pixel based on its intensity. So, what were in the original image two pixels with the same "color", but different intensities end up as different colors. This has been discussed quite intensively in context of the 5DII, which also shows these issues, both on this forum (http://forums.adobe.com/thread/311802?start=0&tstart=0) and on dpreview (http://forums.dpreview.com/forums/read.asp?forum=1032&message=32482073)

             

            If you want to know more about the technicalities, I've written about hue twists here: http://chromasoft.blogspot.com/2009/02/adobe-hue-twist.html

             

            You might say that well designed "hue twisted" profile would keep neutrals neutral, and that's true, but there's a complication - what neutral is varies with white balance, which makes it a judgement call as to how much "twist" you can tolerate close to a hue that might conceivably end up as a neutral. Probably what adobe will need to do to solve this issue (if they decide it impacts enough people to be worth addressing) is to tweak the profile to maintain "untwisted" across a wider range of near neutral colors.

             

            Short Adobe fixing the profiles, you have three possible solutions;

             

            a) Use the earlier profiles which didn't have the twist

             

            b) use a profile generated by Adobe's DNG profile editor (which never have twists)

             

            c) If you want to try and preserve some of the profiles's look, you can also try untwisting the profiles with dcpTool (disclaimer - I wrote dcpTool): http://dcptool.sourceforge.net/ However, be aware that dcpTool can only preserve some part of the look, not all of it.

             

            Sandy

            1 person found this helpful
            • 4. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
              Bill_Janes Level 2

              sandy_mc wrote:

               

               

              If you want to know more about the technicalities, I've written about hue twists here: http://chromasoft.blogspot.com/2009/02/adobe-hue-twist.html

               

               

               

              Short Adobe fixing the profiles, you have three possible solutions;

               

              a) Use the earlier profiles which didn't have the twist

               

              b) use a profile generated by Adobe's DNG profile editor (which never have twists)

               


              Sandy

               

              Sandy,

               

              An interesting post. The link does not work since hue-twist-html is not part of the link, but can be added manually for display of your page. Doesn't Adobe use the DNG profile editor for creating the profiles? Apparently not, from your post.

              • 5. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                sandy_mc Level 3

                Bill,

                 

                Yes, the link somehow isn't quite right - this new forum drives me nuts. Let me try again: http://chromasoft.blogspot.com/2009/02/adobe-hue-twist.html

                 

                To answer your question, it seem fairly certain that Adobe use a very much more sophisticated tool than the profile edititor to generate the various profiles that are suppied with ACR/Lightroom. The profile editor works only on single value of V in the HSV space, but the "twisted" Adobe profiles map at least 16 levels of V

                 

                Sandy

                • 6. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                  photo81 Level 1

                  Thanks for the tip Sandy. I tried your dcptool on some on the D700 "Camera" profiles to untwist them, which does appear to get rid of the green/magenta color casts in problem images. (However some colors, like blue, become oversaturated).

                   

                  However the other problem, which is ugly shadow rendering, is still present. (This can go away by using Adobe standard, or a custom profile generated by DNG Profile Editor after taking a picture of a color chart). Since nobody seems to really be aware of this issue (which affects a sizable fraction of the images I take; almost any image with a shadow), I'm attaching some JPG's of raw files processed from Lightroom with Adobe's "Camera standard" profile, and again with the "Adobe standard" profile. If you just look at and compare shadow areas, I think the problem is obvious.

                  • 7. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                    photo81 Level 1

                    More images (it seems like this forum's attachment upload system is still broken..)

                    • 8. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                      photo81 Level 1

                      Also it seems like browsers have a weird JPEG decode engine (esp in rendering shadow regions), you might have to download the images to properly view them.

                      • 10. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                        jepoma Level 1

                        Dear All,

                         

                        I have a possibly related problem which shows up as chromatic aberration. Please see my blog for examples. It's now bothering me so much that I am, sorry folks, considering switching to NX2. Can someone tell me if what I see is related to the artifacts discussed in this thread and, most importantly, if a fix is coming any time soon ?

                         

                        Yes, I know, I can fix each photograph by hand, but I'd love to be able to start from a correctly rendered photograph, not something I need to work on just so it is the same quality as the JPEG generated in-camera by my D700.

                         

                        Lastly, could Adobe pay Nikon to get the "secret" behind the NEFs and offer us an Lr that renders our Nikon photographs as correctly as they rightly deserve ?

                         

                        Cheers,

                         

                        ++

                        Jepoma

                        • 11. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                          photo81 Level 1

                          I think the CA you're seeing is probably another issue; I notice the same difference between Lightroom and Capture NX. I don't think there's anything wrong with Lightroom, it's just that CaptureNX does some clever automatic CA removal, without having to have the user enter any numbers. Still it would be nice if Adobe could implement an automatic CA reduction feature in a future release.

                           

                          It occurred to me on the files I uploaded earlier, it's kind of hard to see the difference on a webbrowser, especially dark regions on a white background (Firefox and IE do not render dark regions of JPEG's accurately at all).

                           

                          I'm attaching some comparison crops of images converted in Lightroom (top is Camera Standard, bottom is Adobe Standard). I brightened up the images (used curves, dragged the white point to the left), just to exaggerate (and highlight) the issues with shadow rendering. Basically the tonal range has been compressed so you're just seeing the shadow tones.

                          • 13. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                            photo81 Level 1

                            I just found out that if you just re-save a profile from DNG Profile Editor (without doing any color chart calibration), the resulting profile will at least fix the shadow posterization issues - shadows look a lot better. The green/magenta color artifacts still remain though.

                            Without loading a DNG image, just go to Base Profile -> choose external profile -> find whatever Camera profile you need, should be in C:\ProgramData\Adobe\CameraRaw\CameraProfiles or somewhere. Then export the profile and try it.

                            • 14. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                              petr_1

                              Any updates on this problem? That flickr set shows the exact problem what I'm also having -- with or without D700 profiles, doesn't matter, same result.

                              • 16. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                dhazeghi Level 1

                                Still present in ACR 5.6 RC, and still present Lightroom 3 beta.

                                 

                                I must say it is quite disappointing that more than a year after the profiles were first released with these problems, there still does not appear to be an official fix.

                                • 17. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                  jepoma Level 1

                                  Well, Lr 3.0 Beta does not include >all< the features that will be included

                                  in the release version. I still hope Adobe add some form of automatic or

                                  best-guess CA correction. A step in the right direction would be to have the

                                  possibility to flag areas potentialy affected by CA (like one can highlight

                                  areas over- or under-exposed). But, heck, if my D700 can fix CA in-camera

                                  when it produces JPEG, why couldn't Lr do it ?

                                   

                                  Let's keep asking for this !

                                   

                                  Cheers.

                                  • 18. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                    photo81 Level 1

                                    Yes, this is extremely disappointing from Adobe. It's been months since I've reported this issue to Adobe (and sent samples), as even longer since others have noted it. Things like this (and automatic CA correction, for that matter) should be higher priorities for the ACR/LR team than trivial features like slideshows.

                                     

                                    Following up about my last post - I think I've found a fix for the shadow posterization (plus related issues, like reddish shadows). Basically I started with Tom Knoflook's method for making a custom profile (convert some RAW file to DNG, open it in Adobe DNG Profile Editor, choose the base profile you want). Only instead of shooting a custom color target and adjusting the color tables, I just directly saved out the profile (File->Export Profile). You would think that not having done any calibration, the resulting profile should be exactly the same as the original base profile. But in fact, when using these profiles in ACR or LR, the shadow rendering artifacts go away!

                                     

                                    Unfortunately the green/magenta color shifts are still in my D700 files, but I believe this finding further proves that there is a bug. I have included the profiles I "made" as attachments. FYI, on Windows Vista/7, the original Camera profiles are stored in C:\ProgramData\Adobe\CameraRaw\CameraProfiles\Camera. Any user profiles you make can go in another folder, C:\Users\<USERNAME>\AppData\Roaming\Adobe\CameraRaw\CameraProfiles. If you want to try testing my profiles, just put them in that folder and restart Lightroom or ACR. (I'm not sure what the WinXP or OS X paths are but you can figure out).

                                     

                                    I repeated the same thing for some 5D Mark II files and saw the same findings (where the resaved DNG profile editor profile of the Camera profile changes/improves the shadow appearance). Though for that camera the issues are slightly less noticeable. It seems that the shadow rendering issues may be present in the "Camera" profiles for all cameras (not just Nikon D700/D3).

                                     

                                    I will post some image crops showing the differences in the next few posts.

                                    • 19. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                      photo81 Level 1

                                      Put up some comparison crops on this Flickr photoset, from D700 and 5DMk2 files. The top crop is the image rendered in LR with Camera Standard, the bottom one with the profile I resaved from DNG Profile Editor.

                                       

                                      I'll also directly post them (too bad about this 3 image/post limit).

                                      • 21. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                        photo81 Level 1

                                        Some Canon 5D Mark II crops. (I don't have this camera, so these aren't the best shots, the first two are from test shots I took in a store, #3 is from a sample RAW from photographyblog.com)

                                        • 22. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                          Mishamsk Level 1

                                          I second that this is a very important problem which cause many Nikon users to stay on CNX. And I think that is a lot of people and Adobe should pay attention to this!

                                          I switched from Canon to Nikon almost a year ago and I think that LR problems is the biggest regret I have ever since, it would be perfect otherwise!

                                          I`m a Lightroom addict and I still use it even though I have to struggle with profiles frequently but I hope it will be fixed soon!

                                          • 23. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                            pacmanphoto

                                            I am having problems with rough color graduations in shadow areas with my Canon EOS 5D Mark II.

                                             

                                            I tried your solution and resaved the camera profile and I also tried your attached profile (they both look the same, file sizes were a bit different though).

                                             

                                            They doesn't solve the problem though. They just darken the shadow parts a bit, which moves the problem to brighter parts of the picture.

                                             

                                            Even weirder: If I use the resaved profile my JPG looks different in un-colormanaged software like my webbrowser, but the two pictures with the resaved and the original camera profile look exactly the same when I copy and paste them from Friefox to Photoshop!?

                                             

                                            I have got an example to show my problem:

                                             

                                            DPP: http://pacman.heinosoft.com/forum/IMG_7826_dpp.jpg
                                            LR: http://pacman.heinosoft.com/forum/IMG_7826_lr.jpg
                                            LR with Blacks set to 0: http://pacman.heinosoft.com/forum/IM...cks_zeroed.jpg

                                             

                                            As you can see, for example left to the printer, there are these red blotches with very disturbing unclean borders.

                                             

                                            They disappear when I set Blacks to 0 and they are not showing with Canon DPP.

                                             

                                            Is this the problem you guys here are talking about or is this another problem? I'm really not sure, that's why I am asking.

                                            • 24. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                              Panoholic Level 2
                                              function(){return A.apply(null,[this].concat($A(arguments)))}

                                              pacmanphoto wrote:

                                               

                                              As you can see, for example left to the printer, there are these red blotches with very disturbing unclean borders.

                                               

                                              They disappear when I set Blacks to 0 and they are not showing with Canon DPP.

                                               


                                              There are no "red blotches" there. There is a large spot on the side of the cabinet brighter than the rest, caused perhaps by reflection from the printer. If you load both images in PS (the DPP and LR versions) as layers and add a curve to the top increasing the contrast in the shadows, you will see that the "blotch" is there in both versions, only that the different raw conversions caused different contrast on that area.

                                              • 25. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                                pacmanphoto Level 1

                                                Yes I am well aware of that, but the graduation on the Lightroom version looks bad and I can't figure out what to do against it. The graduations are smooth in the DPP version, so it isn't a "blotch".

                                                 

                                                Or hey, look at the borders of the wood. There is reddish ghosting visible in the LR version. Where does that come from?!

                                                • 26. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                                  Panoholic Level 2

                                                  1. The graduations are not smooth in the DPP version either. That "blotch" (the brighter area on the inside of the shelf beside the printer) shows banding, but that's the problem of your camera (look at the vertical strips).

                                                   

                                                  2. I don't see any reddish ghosting. I see brownish, relatively smooth edges of the shelving with the LR version and color blotches with the DPP version (after having added a curve and vastly increased the contrast in the very shadows, like input=76 yielding output=255).

                                                   

                                                  3. The white balance is not the same in the two renderings; I think it is wrong in both. If you make such comparisons, pick WB in both renderings from the same spot. Here the letters on the DVD cases seem to be suitable for that.

                                                   

                                                  Gabor

                                                  • 27. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                                    pacmanphoto Level 1

                                                    The White Balance is the same for both pictures. The one the camera chose automatically - I know it's not the best setting, but I don't think my problems are caused by that.

                                                     

                                                    Here's an animation to clarify things. Frames change each 5 seconds, there are 4 frames.

                                                     

                                                    http://pacman.heinosoft.com/forum/IMG_7826_animation.gif

                                                     

                                                    You can also see how the red blotch is very disturbing in the Lightroom version whereas it looks pretty smooth in the DPP version.

                                                    • 28. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                                      Panoholic Level 2

                                                      1. Re WB: normally it's ok to use the setting of the camera. However, when you are making such comparisons between the results of several raw processors, the WB should be picked, so that one starts from the same point. Anyway, that is not really relevant here.

                                                       

                                                      2. I saw the colored strips in the LR version, which look like color aberration, and the red patch, but I did not think that's your objection. Those are the consequence of "blacks", and that is ok. If someone wants to increase the intensity in post processing so much, then one does not specify a high "blacks". It is a nonsense, that ACR comes with "Blacks 5" as default, and many users don't understand the implications. I changed the default to 0 and increase it only if there is some reason to. However, the "Blacks 0" version of this shot is less contrasty, it looks dull compared to both others when viewed in normal intensity. Perhaps Blacks 1 or 2 would be better.

                                                       

                                                      Anyway, I am not defending LR/ACR, but I don't see any reason to bash them in this case.

                                                       

                                                      Gabor

                                                      • 29. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                                        pacmanphoto Level 1

                                                        1. The white balance is the same for both shots. Namely the white balance the camera chose.

                                                         

                                                        2. Well if I don't set Blacks to 1+ I can't get any reasonable contrast in my photo. Those effects are only gone if Blacks is zero, low values of Blacks help, but depending on the picture the problem is still well visible.

                                                         

                                                        I got it "fixed" with some amount of work to get the photos look like in DPP with Blacks, Brightness and Contrast set to 0 and just the Tone Curve adjusted.

                                                        Looks muuuuch better now for a starting point. I think cutting the Blacks with Blacks set to +5 by default isn't something you should use on every picture.

                                                        If you are interested you can find my preset for the Canon EOS 5D Mark II (maybe it works for other cameras too, since I use Adobe Standard Profile) here: Download

                                                        • 30. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                                          tereza29a

                                                          How professional like you can rely on default camera profiles really puzzles me. camera profiles, in camera raw, are somehow averaged profiles for most cameras. unfortunatelly, perfectly imperfect world of nature, there is no chance that manufactures make exactly same cameras all the time. even one batch can vary. it is called standart deviance in manufacturing and there is no chance that adobe could fix this.

                                                           

                                                          pro solution is buy macbeth colorchecker, download dng profile editor and make your camera profiles. then and only then you get "perfect" results from screen/printer. further more, you need calibrated monitor/lcd which requires colormeter+sw.

                                                           

                                                          otherwise, it could be not only fault of adobe, but as well as you camera, your display.

                                                          • 31. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                                            photo81 Level 1

                                                            Yeah, but this is already a well documented problem that lots of people have observed and complained about. Eric Chan is already working on a fix for D3/D700 Camera profiles in another thread..  and I am already using a calibrated monitor. Also, I have also never really noticed that much variation in color response between different models of the same camera, to be honest... but I might be wrong.

                                                             

                                                            I never said that these profiles are necessarily accurate, they are just intended to match the Nikon rendering. Keep in mind a lot of people are using Capture NX for their RAW conversion and perfectly happy. The Nikon colors, while not perfect, are at least much more accurate and pleasing than the Adobe Standard option. For the vast majority of people, I think having usable Camera profiles would be pretty valuable.

                                                             

                                                            Obviously a custom profile would probably be ideal if you wanted completely customized and accurate colors, but not everyone has access to a Color Checker. Even then, getting a pleasing "generic" profile that works with different lighting conditions is a bit tricky.

                                                            • 32. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                                              tereza29a Level 1

                                                              But it is not problem of Adobe ;-) I don't want to defend Adobe, but you people try to crucifice Adobe and shift irresponsibility of manufacturers on Adobe.  Adobe is not responsible for color profiles. They provide post-

                                                              processing sw. It is responsibility of camera manufactures to provide color profiles.

                                                               

                                                              Because there is most of phofographers who take "i have been there" pictures in JPEG and full automatic setting. Thus generic profiles are enough.Furthermore, generic profiles maybe just for standart daylight of 5400K. Every sensor has got different readings at different light conditions. That is why color calibration simulates 2850K (standart bulb) and 6500K (shadow or so). Then you get "perfect" match. Otherwise I don't know why normal people would need such a accurate setting. Most people would not even notice any difference, unless you tell them. Only pro who knows what does will be unhappy. There is old saying "deal with it or dream about it."

                                                               

                                                              Pro's who need accurate printouts of fine art use WB cards+RAW, calibrated cameras/displays. Then there will be no problems in output. Unless there is some bug in SW which is not related to color profiles.

                                                               

                                                              As far as I am concerned, my Pentax K-7 DNG+WB cards+ACR 5.7(calibrated camera profile with colorchecker+DNG editor) has got "perfect" output. Still, I have not calibrated display so there is little mismatch (I don't prin out yet). There maybe problem with RAW conversion of proprietary RAW formats. NIkon uses encrypted files, I believe, which causes problems. Basically, that is fault of manufacturer of camera. Pentax has got option of DNG/PEF. DNG=Open Source Standart=No problem.

                                                               

                                                              Don't tell me that guy who buys camera for GBP1000, cannot afford color checker for GBP30 ;-)

                                                              • 33. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                                                tereza29a Level 1

                                                                As far as I have studied this thread. All your problems are caused by proprietary RAW formats.

                                                                Unfortunatelly, mainstream camera manufactures are idiots and cannot use one format. Canon must always make different format even on same generation of cameras. NIkon is not different. No wonder that high precise calculation of analogue RAW data is shifting colors and create unwanted artifacts.

                                                                 

                                                                There is two solution though:

                                                                 

                                                                1. moaning at manufacturer's shoulder to use open standarts or release documentation for developers (that's exactly what they don't want).

                                                                 

                                                                2. related to point 1. willingness to pay more for Adobe products because once they buy license for proprietary formats, price will double.

                                                                 

                                                                2. use proprietary manufacturer's sw (which is rubbish and matches poor JPEG output of camera engine)

                                                                 

                                                                conclusion:

                                                                 

                                                                it is everlasting circle of greed and incompetence of camera manufacturers and people around. I have no idea what "secrets" try Canon/Nikon hide in their RAW formats. As far as I have observed it is incompetence to get clean output from their sensors without heavy postprocessing. You can see propaganda of Canon/Nikon how "best" their product are. But don't compare RAW output without postprocessing. I noticed that Canon/Nikon covers their sensor noise by heavy blending. No wonder that their RAW formats cause artifacts in 3rd party SW.

                                                                • 34. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                                                  Bert Nase Level 1

                                                                  Oh well, that concludes Nikon is responsible to deliver camera profiles to make ACR work as perfect as Capture NX2? What I've seen so far LR3b2 with ACR 6 comes close to what NX2 can deliver. And there are a lot of free raw converter out there that could it better then ACR though you have to use them in batch but what counts is the result. So who if not Adobe is responsible for the results of ACR?

                                                                  • 35. Re: D700 Camera Profile Artifacts (still present in ACR 5.5?)
                                                                    tereza29a Level 1

                                                                    Well known issue is Nikon's encrypted proprietary RAW files. Unless, NIkon will be willing to release full documentation. 3rd party RAW converters cannot process file successfully. You are barking at wrong tree. You have to force NIkon to use DNG for example. They just tell you, use our perfect SW.

                                                                     

                                                                    Perhaps owner of camera who suppose to buy colorchecker and make own profiles if he wants perfect match. Paper manufacturers provide color profiles for their paper to get most of it. I don't see reason why camera manufacturers cannot deliver color profiles for their products as well. They know their cameras best. I don't have artifatcs with my camera (Pentax K-7). It may also be caused by proprietary RAW  file. Try ufRAW+GIMP and see if you get same result.