1 2 Previous Next 52 Replies Latest reply on Jan 31, 2009 9:36 PM by Panoholic

    RAW to DNG just some simple advice please

    Robbie Dahl Level 1
      I would like some advice on converting RAW to DNG. I have read on line that it is a good practice to convert all my RAW images to DNG after uploading them. I am using Adobe Lightroom 2.2. When I use "Convert to Linear Image", my files double in size. Should I be using "Preserve RAW Image" ? What is best practice and why? Should I be "embedding original RAW image"? Should I leave it as the one Panasonic/Lumix RAW format?
      I am really confused and would appreciate any advice?
      I have tried reading all the threads on this site and they are just too technical and confusing for me at this stage. I just want to know that I am following good practice and treating my images in the best possible way. I have much to learn about using Lightroom but would just like to get past what formats to actually use.
      Should I be doing all my edits on the jpeg instead and not even bother with the RAW?
        • 1. Re: RAW to DNG just some simple advice please
          Ann Shelbourne Level 1
          The only reason to use "Preserve RAW Image" when you save as .dng is if you are likely to want to re-process the original RAW capture in your camera manufacturer's own software.

          If you intend to use only ACR to make future edits to your raw file, you can dispense with "Preserve RAW Image" and enjoy the benefit of much smaller file sizes.
          • 2. Re: RAW to DNG just some simple advice please
            Panoholic Level 2
            > I have read on line that it is a good practice to convert all my RAW images to DNG after uploading them

            Let's keep things as straight as possible. There are several issues here.

            1. Archivation: this has *nothing* to do with the workflow; if someone say differently, he is lying.

            There is only *one* original: that is, what you get from the camera (if that is truly raw or not is irrelevant, for it is not under your control).

            So, if one wants to archive the *original*, then that is the CR2 or NEF, as it came out of the camera (*not* out of DPP or Nikon Capture).

            2. Working version.

            This depends on your choice of raw converter and of the following steps.

            > I have tried reading all the threads on this site and they are just too technical and confusing for me at this stage

            Well, that's the
            i state of the art regarding raw image processing.
            Are you sure you need to work with raw data? Perhaps your thought
            > not even bother with the RAW

            was justified at this point of your image processing experience.
            os the best adice at this point of your experience.
            • 3. Re: RAW to DNG just some simple advice please
              Robbie Dahl Level 1
              Thank you for that response Ann. As far as I am aware Panasonic does not provide me with any special software so I won't be using that option.
              My RAW images are 19.3mb and when converted to DNG using the "Convert to Linear Image" option the resulting DNG file is 36.5mb. Does this mean I should just stick with the RAW file or am I doing something wrong in the conversion.
              • 4. Re: RAW to DNG just some simple advice please
                Robbie Dahl Level 1
                G Sch - just got your response and thank you for those thoughts. I am hoping that I will eventually be able to use Lightroom to its full potential so I am happy at this stage to work only on the Jpeg copies of my photos, but would like, in the future, to be able to reprocess some of my special images using the RAW images. Should I save them as RAW or DNG? Does it matter? Sorry to ask such basics on this forum but Lightroom is supposed to be good for professionals and novices like me as well?
                • 5. Re: RAW to DNG just some simple advice please
                  Panoholic Level 2
                  > Should I save them as RAW or DNG?

                  Archive always the *orginal* (in this case the CR2). Note, that DPP is very good for previewing, but if you make any trial adjustments, it wants to change the CR2 file.
                  b Do not allow DPP saving any changes
                  before your archived the original. Use DPP or ZoomBrowser for selection only.
                  • 6. Re: RAW to DNG just some simple advice please
                    Robbie Dahl Level 1
                    I am shooting with a Panasonic Lumix FZ50, but I assume the advice given for Canon formats apply. Therefore I will save all my RAW images as they upload to my hard drive, completely unchanged and unprocessed.(Also copy them as a backup on an external hard drive, of course.) Thank you so much for taking the time to address my concerns.
                    • 7. Re: RAW to DNG just some simple advice please
                      JimHess-8IPblY Level 3
                      As far as editing is concerned, it doesn't matter whether you work on the original raw image or on a DNG file. The image data is precisely the same, and you can achieve identical results from either one. One of the arguments I've heard in favor of DNG is that it is an open file format that you will always be able to turn to, at least as far as Adobe is concerned. Some people advocate archiving the original raw files and then creating DNG copies on which to do all of their editing. Others have indicated that they convert to DNG and then discard the original raw images. Unfortunately, there isn't a right or wrong choice here. You have to decide what you are comfortable with, and then stick with it.

                      One of the reasons I like working with DNG is that when I'm using Photoshop/ACR I don't have to worry about all of the XMP sidecar files. I know they don't take up a lot of room. I just personally prefer not to have to worry about them. With DNG files all of that information is stored in the header section of the DNG file.

                      As has been stated previously, one of the major reasons for keeping the original files is to enable you to work on them with proprietary software provided by the camera manufacturer. That software is not going to recognize DNG files.

                      As you can see, there are a number of considerations that you have to make. Decide what will work best for you. There is no right or wrong answer. Unfortunately, this is a choice YOU are going to have to make for yourself.
                      • 8. Re: RAW to DNG just some simple advice please
                        Panoholic Level 2
                        > one of the major reasons for keeping the original files is to enable you to work on them with proprietary software provided by the camera manufacturer

                        NO!

                        The reason to keep the original file is
                        b to be able to start from point zero,
                        i.e. skipping whatever unwanted or nonsensical effects some previous steps have caused.
                        • 9. Re: RAW to DNG just some simple advice please
                          Robbie Dahl Level 1
                          Thank you Jim,
                          You have made a lot of what I have been reading start to make sense. I didn't expect there to be such a range of options.Yes, I will do as you say and just think about what works best for me and then go with that. Really, really appreciated.
                          • 10. Re: RAW to DNG just some simple advice please
                            JimHess-8IPblY Level 3
                            G Sch,

                            YES! Notice that I said "one of the reasons". I didn't say it was THE reason. The reason I stated has been voiced by a number of people including those who teach Photoshop for a living. I'm not saying your reason is wrong. But it is not THE reason either.
                            • 11. Re: RAW to DNG just some simple advice please
                              Ann Shelbourne Level 1
                              Re: #8:

                              A .dng saved with some existing ACR edits (an embedded sidecar in effect) can be easily reverted to it's original form in ACR.

                              The only thing that you lose by omitting to choose "Preserve RAW Image" are the special Camera Effects Settings which you may have activated and which ONLY affect images captured as JPEGs but which can be used to affect RAW files IF and ONLY if you subsequently edit those RAW files in the Camera Manufacturer's own proprietary Raw Converter software.

                              ACR cannot recognise nor use those Camera Settings so there is no point in choosing "Preserve RAW Image" if you plan on doing any future editing in ACR.
                              • 12. Re: RAW to DNG just some simple advice please
                                Panoholic Level 2
                                > A .dng saved with some existing ACR edits (an embedded sidecar in effect) can be easily reverted to it's original form in ACR

                                The "original form in ACR" is not the original form of the image data.

                                > The only thing that you lose by omitting to choose "Preserve RAW Image" are the special Camera Effects Settings

                                The only thing that you lose is simply the
                                i original data.

                                One may discuss about the
                                i relevance
                                of this fact, but not about this
                                i being a straighforward fact.

                                Those, who think the DNG is equivalent to the original raw image need to read for example this: http://forums.dpreview.com/forums/read.asp?forum=1032&message=30746152
                                • 13. Re: RAW to DNG just some simple advice please
                                  Ann Shelbourne Level 1
                                  I did read it and I still don't understand how it is relevant if I shoot RAW; edit in ACR; Convert the saved ACR data to DNG; and reopen the DNG in ACR.

                                  All that I have to then do is to revert ALL ACR settings back to ACR Default and choose a Camera Profile to be back exactly where I was before I made any edits in ACR or Converted to DNG.

                                  Or I could go further and strip all settings in ACR to zero and start again from there.

                                  What have I "lost" that is of any value except for the in-camera picture-editing settings which have no effect on my Nikon D3 RAW images unless I choose to use Nikon's Capture NX2 (which I don't!).
                                  • 14. Re: RAW to DNG just some simple advice please
                                    JimHess-8IPblY Level 3
                                    G Sch,

                                    I know that this has been a source of extensive discussion with you in the past. I understand that from your point of view the only way to start from "ground zero" is to have the original raw file. And that is your reason for keeping those files. Adobe has held the position that the image data is exactly the same in the DNG file. While you disagree with this, my experience has been that having the DNG file is close enough to "ground zero" for me. I have been able to clear Camera Raw settings and start over to my satisfaction. If there is a difference (as you maintain) that difference is so minimal that it doesn't concern me. Like I said, everyone has their own reasons for using/not using DNG and whether or not it is necessary to retain the original raw images. You have your reasons, I have mine. Neither one of us is right or wrong.
                                    • 15. Re: RAW to DNG just some simple advice please
                                      Ann Shelbourne Level 1
                                      I have just tried the time-honoured Photoshop "Difference" Test.

                                      I saved a DNG file out of ACR as a 16-bit 300 ppi Prophoto RGB .psd.
                                      I then did the same thing to the original RAW file from which I had made the DNG Conversion.

                                      I opened both in Photoshop CS4 and superimposed one over the other in separate layers.

                                      I changed the top layer to "Difference".

                                      Even at a 3200% Zoom, the image displays as solid Black and there is not a shimmer of white to be seen in other words, there is NO difference between the DNG file and the original RAW file as edited in ACR 5.2.

                                      Well there IS one not-so-small difference:
                                      the DNG file occupies 12.9 MB of disk space while the original RAW takes 24.33 MB.
                                      The PSD files each fill 69MB.
                                      • 16. Re: RAW to DNG just some simple advice please
                                        MadManChan2000 Adobe Employee
                                        Robbie, what is the file size when you check the Preserve Raw Image option instead of the Convert to linear option?

                                        Regarding image preservation, for mosaiced DNGs produced by the DNG Converter, Lightroom, and Camera Raw from a non-DNG raw file, the resulting image data (i.e., the actual mosaiced pixel data) is preserved bit-for-bit. The only difference is the encoding: mosaiced DNGs are stored using Lossless JPEG (essentially delta Huffman compression). The difference is the container and the surrounding metadata. The issue that Gabor points out is an error in the metadata, not the image data (i.e., the image data is intact, but the metadata is wrong, hence the visual results are wrong). Obviously, in practice you want to get all the pieces correct ...
                                        • 17. Re: RAW to DNG just some simple advice please
                                          Ann Shelbourne Level 1
                                          Could Gabor's problem be due to the way that his camera records the metadata originally?

                                          He is using a Canon I believe; and I am using a Nikon D3.
                                          • 18. Re: RAW to DNG just some simple advice please
                                            Robbie Dahl Level 1
                                            Hi Eric,
                                            1 - The file size of my RAW image off the camera is 19.3mb. (The attached jpeg is 2.37mb.)
                                            Convert this to DNG "Convert to Linear Image", Compressed(Lossless) is 39.3mb.
                                            2 - The file size coverted to DNG "Preserve Raw Image" is 9.52mb.
                                            3 - DNG with "Embed Original" is 24.6mb.
                                            Not sure I have followed all that has been said so far but I will keep all your comments and decide what to do.
                                            Would appreciate your comments on the file sizes Eric.
                                            Regards, Robbie
                                            • 19. Re: RAW to DNG just some simple advice please
                                              Panoholic Level 2
                                              One has to have boundless faith when throwing away the *original* of such complex "bunch of information" as a raw image. I don't have that faith even in my own programs, and of course even less in Adobe's. Furthermore, I trust the camera manufacturers' software abilities even less (close to zero). I have seen several errors already, and I see it possible, that certain errors could be corrected in the original format but not after the conversion.

                                              Eric's statement

                                              > the resulting image data (i.e., the actual mosaiced pixel data) is preserved bit-for-bit

                                              is correct in most cases, but not always. I see no reason to divert this thread into a technical discussion. Instead, I uploaded two DNG files (from Nikon D3), created from the very same NEF, by two different versions of DNG. Those, who have unlimited faith in the DNG conversion should take a look at them in LR/ACR.

                                              http://www.panopeeper.com/Download/D3_V4.3.0.79.dng
                                              http://www.panopeeper.com/Download/D3_V5.2.0.65.dng
                                              • 20. Re: RAW to DNG just some simple advice please
                                                Ramón G Castañeda Level 4
                                                Ann,
                                                >I have just tried the time-honoured Photoshop "Difference" Test.

                                                Not disputing the possible veracity of your conclusion, but the time-honoured Photoshop "Difference" Test should be reviewed.

                                                Even Bruce Fraser didn't adhere to that test.

                                                If there should be a very small difference in a given pixel, the otherwise 255,255,255 pixel would then appear as 255,254,255 or 254,255,255 or something similar, which you would still see as black on your monitor, regardless of magnification.

                                                Two better, more definitive testing methods would be (a) one suggested by Bruce Fraser himself (I'll post it if and when I find the exact text), and (b) a new (to me) method suggested by someone in the Color Managament and Photoshop Windows forums, which follows:

                                                (NOTE: only the methodology is of interest and pertinent, not the questionable context in which it has brought up and used.)

                                                * 1) Open the two images to be compared in Photoshop

                                                * 2) Move one image as a layer over the other one

                                                * 3) select "Difference" as blending mode in the layers palette

                                                * 4) now the whole image should appear seemingly black on the monitor

                                                [So far this is the traditional, "time honored" method.]

                                                * 5) select the magic wand tool with these settings: Tolerance: 0/ Anti-alias: no/ Contiguous: no/ Sample All Layers: yes

                                                * 6) click somewhere into the formerly gray area

                                                Explanation: you just selected all completely black pixels (0,0,0) i.e. all pixels that are identical in both layers.


                                                * 7) you should see "marching ants" forming rectangular patterns

                                                * 8_) invert the selection (Shift Command I)

                                                Explanation: the selection now covers all the other pixels, i.e. all pixels which are different between both layers.


                                                * 9) create a new empty layer and select it in the layers palette

                                                * 10) set the foreground color to white

                                                * 11) fill the selection with white (Alt+backspace on Windows, accordingly on Mac)

                                                * 12) set the blending modes of all layers back to normal

                                                Explanation: you now see all identical pixels in their respective color and all different pixels in white.


                                                This method is a lot more sensitive than the traditional one which stops at step #4 above.
                                                • 21. Re: RAW to DNG just some simple advice please
                                                  Ann Shelbourne Level 1
                                                  Unfortunately Gabor's v.4.3 DNG file appears to be corrupt.

                                                  The 5.2 version opens and looks reasonably OK in ACR 5.2 with the Adobe or Camera Standard Profiles.

                                                  However, I notice that it was shot at the extended ISO of 25,600 under mixed lighting so it is perhaps not the best example to use.

                                                  It was also saved with only the ACR 5.2 Default settings.

                                                  So, yes, the DNG opens and apparently retains its original metadata including the WB.
                                                  • 22. Re: RAW to DNG just some simple advice please
                                                    Ann Shelbourne Level 1
                                                    Interesting Ramón. I will try it when I have time.
                                                    • 23. Re: RAW to DNG just some simple advice please
                                                      Panoholic Level 2
                                                      > Unfortunately Gabor's v.4.3 DNG file appears to be corrupt

                                                      What does this "appear" mean? Can't your ACR open the file? Do you receive some error message? My one is 4.6.0.30 and it can process the file.

                                                      > I notice that it was shot at the extended ISO of 25,600 under mixed lighting so it is perhaps not the best example to use

                                                      I don't understand this. Is it acceptable to corrupt noisy images when converting in DNG?
                                                      • 24. Re: RAW to DNG just some simple advice please
                                                        Ann Shelbourne Level 1
                                                        I am using CS4/ACR 5.2 on a Mac running on OSX 10.4.11

                                                        Your 4.3 DNG opens in ACR but it is filled with extreme noise over the lower part of the image which is not present in your 5.2 DNG.

                                                        But might it not be a good idea to start the experiment with a better quality photograph?

                                                        However, DNG Converter 5.2 did not corrupt the image even if it is noisy.
                                                        • 25. Re: RAW to DNG just some simple advice please
                                                          Ann Shelbourne Level 1
                                                          This is how your 4.3 version Previews in Bridge: <br /> <br /> <a href="http://www.pixentral.com/show.php?picture=108SV5vyQYvk4o7AQx19L72XVObF1" /></a> <img alt="Picture hosted by Pixentral" src="http://www.pixentral.com/hosted/108SV5vyQYvk4o7AQx19L72XVObF1_thumb.jpg" border="0" /> <br /> <br />Compared with the 5.2 version: <br /> <br /> <a href="http://www.pixentral.com/show.php?picture=1Swtlc1oFJhQKwkMDNnQKsysRBQZH1" /></a> <img alt="Picture hosted by Pixentral" src="http://www.pixentral.com/hosted/1Swtlc1oFJhQKwkMDNnQKsysRBQZH1_thumb.jpg" border="0" />
                                                          • 26. Re: RAW to DNG just some simple advice please
                                                            Panoholic Level 2
                                                            > Your 4.3 DNG opens in ACR but it is filled with extreme noise over the lower part of the image which is not present in your 5.2 DNG

                                                            Ann, that's just the point.

                                                            b The older version of the DNG converter corrupted the image data.

                                                            I have several examples for this error. The background is, that Nikon introduced a new twist in the lossy compression of the D300 and D3 data; one could say it is "lossy in two dimensions" (the lossily compressed data undergoes another lossy step). Adobe did not figure out initially the correct decompression. Only very noisy images are affected, typically from ISO 6400; such images have been corrupted months long.

                                                            Those, who trusted the DNG conversion and threw away their NEFs can forget about the images; it is not the question of metadata.

                                                            It goes even worse: those, who started out with converting the NEFs in DNG before even looking at them in CNX must have thought that their images were garbage, while they may have been usable, like the second version of the posted image shows.
                                                            • 27. Re: RAW to DNG just some simple advice please
                                                              Ann Shelbourne Level 1
                                                              Was that older version of Adobe DNG Converter a Beta version?

                                                              I have only just started to use the DNG Converter and I have not run into problems with the current version 5.2.

                                                              Perhaps it would be wise if Adobe offered the option to Convert to DNG with embedded ACR metadata but without compression?
                                                              • 28. Re: RAW to DNG just some simple advice please
                                                                Yammer Level 4
                                                                I've read all the arguments (including those in Bruce/Jeff's book) but my worksflow still doesn't involve DNG, and I'm still not sure why it should.

                                                                My Nikon D300 produces 14-bit lossless compressed NEFs. These are copied to my working raw folders, which are regularly backed-up to offline storage. ACR writes metadata to sidecar files. The NEF files are never written to, and so always remain as they were initially intended. Sidecar files can always be deleted if ever necessary. I like that they are separate. It works for me.

                                                                The only reason I would ever need to convert to DNG, as far as I know, would be the obsolescence argument - which I still can't get my head around. Somehow, in the future, NEF won't be supported and DNG will. I'm finding it difficult to picture that.
                                                                • 29. Re: RAW to DNG just some simple advice please
                                                                  Panoholic Level 2
                                                                  > Was that older version of Adobe DNG Converter a Beta version?

                                                                  No, and it was not the only faulty version.

                                                                  Anyway, if you are not using the lossy compression (you should not anyway), then this error can not have affected you.

                                                                  > Perhaps it would be wise if Adobe offered the option to Convert to DNG with embedded ACR metadata but without compression?

                                                                  It has nothing to do with the compression in DNG but with the compression in NEF.
                                                                  • 30. Re: RAW to DNG just some simple advice please
                                                                    Ann Shelbourne Level 1
                                                                    No I don't use lossy compression. And I don't use any compression at all in my original 14-bit RAW captures.
                                                                    • 31. Re: RAW to DNG just some simple advice please
                                                                      Panoholic Level 2
                                                                      > I don't use any compression at all in my original 14-bit RAW captures

                                                                      Well, if you have enough CF cards and if you don't archive the NEFs anyway, then why not. However, there is nothing wrong with the lossless compression in NEF.
                                                                      • 32. Re: RAW to DNG just some simple advice please
                                                                        Ramón G Castañeda Level 4
                                                                        >I've read all the arguments (including those in Bruce/Jeff's book) but my workflow still doesn't involve DNG, and I'm still not sure why it should.

                                                                        Ditto here.

                                                                        I've tried all the advice given to me by Jeff as well, and DNGs continue to take significantly longer to open than raw files.

                                                                        The "archival" argument escapes me altogether also. As long as Adobe software opens my raw files, why should I worry? And if ACR stops supporting the files from my camera in a catastrophic situation, what guarantee is there that it or any other software will continue to support DNGs under the same circumstances?

                                                                        Given Gabor's findings, I'm even less inclined to use DNGs.
                                                                        • 33. Re: RAW to DNG just some simple advice please
                                                                          Panoholic Level 2
                                                                          > DNGs continue to take significantly longer to open than raw files

                                                                          I guess your original raw files are not compressed. The compression adopted in DNG is very processor intensive; people often do not want to believe, that the required computing time is more than reading the longer data takes.
                                                                          • 34. Re: RAW to DNG just some simple advice please
                                                                            Ramón G Castañeda Level 4
                                                                            Thanks for the further explanation, Gabor. Makes sense.
                                                                            • 35. Re: RAW to DNG just some simple advice please
                                                                              Robbie Dahl Level 1
                                                                              I did not expect such a debate, but it has been extremely enlightening and I think all you guys are amazing.It is so good to be able to get such informed comments on items like this. Anyway, I have made one decision as a result of all this and that is that I will be leaving my files in their unprocessed RAW state and not converting them to DNG. It seems there is no real reason to convert and I don't have to worry about all the technical stuff. Good chance that if, in the distant future, my RAW files are no longer able to be read by any software, I will have some warning and an opportunity to convert them to whatever is the go at the time. Chances are there will be even better options by then.
                                                                              • 36. Re: RAW to DNG just some simple advice please
                                                                                Ramón G Castañeda Level 4
                                                                                Very wise decision and post in my view, robbie.
                                                                                • 37. Re: RAW to DNG just some simple advice please
                                                                                  Yammer Level 4
                                                                                  Ha ha! I just re-read the title of this topic.
                                                                                  • 38. Re: RAW to DNG just some simple advice please
                                                                                    Ramón G Castañeda Level 4
                                                                                    Re #20:
                                                                                    >more definitive testing methods would be (a) one suggested by Bruce Fraser himself (I'll post it if and when I find the exact text)

                                                                                    This is the text I saved a text clipping some four years ago:

                                                                                    A better way of comparing images with identical pixel dimensions is to use Apply Image>Subtract with an offset of 128.

                                                                                    Difference only shows pixels that are lighter in the source than in the target (or maybe it's the other way aroundI forget) where Subtract with Offset 128 shows differences in both directions.

                                                                                    Pixels that are identical in both images come in as RGB 128 gray, those that are different come in at a value that exactly reflects how different they are.

                                                                                    It also makes it much easier to spot subtle differences
                                                                                    • 39. Re: RAW to DNG just some simple advice please
                                                                                      Ramón G Castañeda Level 4
                                                                                      Also re #20:
                                                                                      >6) click somewhere into the formerly gray area

                                                                                      This refers to an image of a Color-Checker type of card in the original image that had a wide gray border around it. The test, therefore, requires a pure gray image in the image, something highly unlikely to change, in order for the magic wand to select all pure-black images (255,255,255). Such a border can easily be created around an image by increasing the canvas size and filling the newly created space with pure gray (128,128,128).

                                                                                      Bruce Fraser's method, outlined in #38, has the advantage of not requiring this.
                                                                                      1 2 Previous Next