18 Replies Latest reply on Dec 4, 2014 8:37 AM by areohbee

    Converting RAW to DNG! What are the advantages / disadvantages?

    Abambo Level 4

      I have Canon cameras and shoot RAW, but the question itself is surely also valid for Nikon, Sony or other brand-shooters.

       

      When importing my pictures to Lightroom, I could convert them to DNG but didn't do this up to now, because I do not know what I loose. I suppose that I loose the ability to process my files with DPP, but I will not do that anyhow. I can appreciate the advantage of having the file in an open documented format, where even I could easily write a reader for it. So the answer should not be what I can easily find in the Adobe introduction into DNG. I have also the DNG specification in front of me, and I can read it, because I once wrote a program to read TIFF files. I understand that I could add the original RAW image stream, but I do not want using up the space wasted for this. I know, that all development parameters used for the specific RAW image can be written with the image file instead of having those stored in a database or a side-car-file.

       

      But what I did not find until now a technical explanation about what happens during the convert. Who can enlighten me? Where do I find the missing technical explanations?

       

      Thanks in advance!

       

      Lucien.

        • 1. Re: Converting RAW to DNG! What are the advantages / disadvantages?
          dj_paige Level 9

          There are dozens of threads on this issue in this forum, plus elsewhere on the Internet.

          • 3. Re: Converting RAW to DNG! What are the advantages / disadvantages?
            JimHess Adobe Community Professional & MVP

            I don't know where you will find a technical brief that will give you the information you are looking for. The non-technical "technical" explanation is that the raw image data gets copied to the DNG file format. That image data is unaltered. The DNG file format standardizes how information is stored. I know that isn't what you want. And I don't know that you can get it.

            • 4. Re: Converting RAW to DNG! What are the advantages / disadvantages?
              Lr5user-pt Level 2

              The only disadvantage I can think of is the loss of compatibility with other converter software, DPP mainly, but DNG support is not widespread with other vendors, at the moment.

              The advantages are also not many. Yes, it’s an open format, yes, you can keep the development steps inside the file, but these steps are software specific (Adobe ACR/LR specific in this case) so without any meaning to other software. There is a modest gain in file size. So not much of an advantage, IMO.


              I believe the DNG SDK documentation has all the details, but I do not have it in this computer. In simple terms, during conversion to DNG all data is basically rearranged to the new format and losslessly compressed. Raw sensor data is transposed bit-by-bit, MakerNotes metadata is also moved untouched and uninterpreted, and the rest is simply moved to the corresponding DNG fields. So, all data is strictly preserved without any modification. Now, this makes me wonder why camera makers refuse to adopt DNG as an option in their software...

              • 5. Re: Converting RAW to DNG! What are the advantages / disadvantages?
                areohbee Level 5

                My opinion: if it ain't broke, don't fix it.

                 

                i.e. if your software reads your raw files in the format they are in, then reformatting them (e.g. converting to DNG) is like solving a problem that you don't have..

                 

                Put another way, the pros/cons of dng vs. original format fall into 2 categories:

                * deal breakers (your software won't read the current format).

                * escoteric (if you don't know about them, they probably don't matter much, if at all..).

                 

                The main problem with converting to DNG is:

                * it's not wise (in my opinion) to discard the originals after conversion, and so after conversion you now have 2 copies that satisfy the same purpose as the 1 copy you had to start out with - extra complication/storage..

                 

                pros/cons also fall into these 2 categories:

                * practical today

                * philosophy and future projection..

                 

                Main practical advantage of DNG:

                * Edited/embedded preview may be readable from the raw file, so you don't necessarily have to have an exported version of it to see edits outside Lightroom (but you MUST have DNG-preview-smart software to take advantage).

                 

                Main practical difference between DNG and original:

                * xmp metadata is embedded in DNG, whereas it's stored in a separate "sidecar" file alongside non-DNG raw.

                Whether it's an advantage or disadvantage depends.. (on person, situation, .. ).

                 

                Main practical advantage of original:

                * Readable by more raw-savvy software, including manufacturer's.

                 

                Philosophical advantage of DNG:

                * Some people believe that all raw files should be kept in the same format, and that format should be DNG. Those people believe DNG will survive long after the original formats have died and gone to wherever dead formats go.. They often cite a Kodak raw file format which died and left some of the "less conscientious" Kodak users (those who didn't convert format before they no longer had access to a machine to do it) in the cold.

                 

                Bottom line: people don't convert to DNG because their original format allows them to do all they want.

                people do convert to DNG because of practical advantages, or because of a warm fuzzy feeling they get, like they are future-proofing, or just "doing a right thing"..

                 

                If you do convert to DNG, then more power to ya. If you change your mind you can go back using a plugin called UnDNG (originals must still be available).

                 

                If you don't convert to DNG, then do consider keeping all your photos in a common tree. If your camera manufacturer goes out of business, and their format will eventually die too, then convert *before* you no longer have a machine to run the conversion.

                 

                PS - To me, it's like asking if you should convert your lossless .flac audio files to lossless .mp4. Answer: if you can't do what you want with your flacs, then yes, otherwise: only if it's bugging you on a philosophical level.. - mp4 is cooler in some circles (e.g. those who think newer is better), flac is cooler in others (e.g. those who are proud of using Linux instead of Mac or Windows..).

                 

                Technical-wise, in general:

                * raw data is copied "verbatim" (transformed, but in a fashion that the original can be re-obtained exactly).

                * metadata is divided into 2 types:

                o Standard

                o Non-standard

                standard metadata is translated to standard accessible form (e.g. exif).

                non-standard metadata is copied over as "black-box" - inaccessible today, but present, and maybe accessible tomorrow (e.g. focus points).

                 

                Cheers,

                Rob

                • 6. Re: Converting RAW to DNG! What are the advantages / disadvantages?
                  Abambo Level 4

                  Very helpful answer! I searched and did not find!

                  • 7. Re: Converting RAW to DNG! What are the advantages / disadvantages?
                    Abambo Level 4

                    Thanks, but that's exactly the kind of info that I would give too. Now the problem is: what gets altered/discarded?

                    • 8. Re: Converting RAW to DNG! What are the advantages / disadvantages?
                      Abambo Level 4

                      ...and it makes me wonder why there is an option to keep the RAW file as is in the DNG container!

                       

                      I will have a look into the DNG SDK documentation! Thanks.

                       

                      To answer your question: The big camera makers have their own format giving them total control on their file. But yes, why does nobody of the big 5 implement this format on their cameras? The "small" middle format camera and back makers are grateful to embrace DNG, as this frees valuable resources for them.

                      • 9. Re: Converting RAW to DNG! What are the advantages / disadvantages?
                        Abambo Level 4

                        Rob Cole wrote:

                         

                        My opinion: if it ain't broke, don't fix it.

                         

                        That wasn't the question... . I want to know what happens, so that I can decide on how to proceed... And getting that "small" size/performance advantage could be a good motivation...

                         

                        Main practical difference between DNG and original:

                        * xmp metadata is embedded in DNG, whereas it's stored in a separate "sidecar" file alongside non-DNG raw.

                        Whether it's an advantage or disadvantage depends.. (on person, situation, .. ).

                        It's always an advantage, when moving files...using standard tools (backup)

                        Philosophical advantage of DNG:

                        * Some people believe that all raw files should be kept in the same format, and that format should be DNG. Those people believe DNG will survive long after the original formats have died and gone to wherever dead formats go.. They often cite a Kodak raw file format which died and left some of the "less conscientious" Kodak users (those who didn't convert format before they no longer had access to a machine to do it) in the cold.

                        Yes that is an advantage of DNG. Even if all companies die, I still can create a program to read my files...

                        If you do convert to DNG, then more power to ya. If you change your mind you can go back using a plugin called UnDNG (originals must still be available).

                        Double space needed... takes away all advantage except the "it's a standard and well documented format" mantra!

                        PS - To me, it's like asking if you should convert your lossless .flac audio files to lossless .mp4. Answer: if you can't do what you want with your flacs, then yes, otherwise: only if it's bugging you on a philosophical level.. - mp4 is cooler in some circles (e.g. those who think newer is better), flac is cooler in others (e.g. those who are proud of using Linux instead of Mac or Windows..).

                        I would ask the same questions: What will I loose or get by doing that. It's all about knowledge. And I am happy to learn from the experts.

                        Technical-wise, in general:

                        * raw data is copied "verbatim" (transformed, but in a fashion that the original can be re-obtained exactly).

                        * metadata is divided into 2 types:

                        o Standard

                        o Non-standard

                        standard metadata is translated to standard accessible form (e.g. exif).

                        non-standard metadata is copied over as "black-box" - inaccessible today, but present, and maybe accessible tomorrow (e.g. focus points).

                        This may be the answer to the question! Where are your sources?

                         

                        Thanks for taking the time and trying to give a complete answer...even that I'm not quite satisfied with it, I feel that it covers a wide range of the subject. Unfortunately, most of the philosophical aspects covered in your answer are common knowledge. And I didn't want to ask a beginners question...

                         

                        Regards,

                        Lucien.

                        • 10. Re: Converting RAW to DNG! What are the advantages / disadvantages?
                          Abambo Level 4

                          Thanks, I did find this article during my research. It's a good non-technical advice. But I wanted to go further into the subject.

                          • 11. Re: Converting RAW to DNG! What are the advantages / disadvantages?
                            areohbee Level 5

                            Feierwoon wrote:

                             

                            That wasn't the question... .

                            Based on the title of the thread and your initial post, it seems like your question had a lot of

                             

                            "What are the advantages / disadvantages?"

                             

                            in it, and only a little

                             

                            "What happens at a technical level when converting"

                             

                            But if you want more detailed info about conversion, and you have the aptitude to understand, I recommend doing what Lr5user-pt recommended:

                             

                            Download the DNG SDK and dig in - all your questions will be answered (and if not, you can also ask DNG-specific technical questions in the DNG forum).

                             

                            As far as my "sources" regarding the technical summary I presented, they are many and varied, none of which are the DNG SDK. In other words, my knowledge is based more on experience over the years - I don't know all the details under the hood..

                             

                            Some experiments to consider doing:

                            * exiftool (-X) a proprietary raw file, then convert to DNG and repeat.

                            * open a proprietary raw in a DNG-supporting software of your choice and inspect metadata, then open the converted DNG in the same software and re-evaluate. Compare to proprietary raw opened in manufacturer's software, and what you see via exiftool.

                             

                            Tom Hogarty (Lightroom project manager) and others who know have said: "DNG converter discards no metadata", and I believe them, but haven't verified for myself. Yet once proprietary metadata is in the DNG, most (all?) software will no longer decode it. A prime example is focus points - presumably they're there, but once converted, focus point feature is no longer supported. The amount of work required to support converted focus point data is unknown (to me), but as far as I know, no software has ventured to do it. If you want focus points, don't convert.

                             

                            If you want the truth, I think for most people it's "not wise" to convert to DNG. Why? because unless you already know why you are converting (and need to convert to satisfy your goals), or you already know why you must NOT convert (e.g. so you can open the files in mfr.software, and see focus points..) the pros and cons are likely to be a wash, and now you have twice the number of raw files, unless you discard your originals, which would be REALLY "not wise", in my opinion. If you find the notion of "openly documented format" sufficiently compelling, then, ya know, more power to ya (I don't judge..), but there is no guarantee that DNG will outlast NEF (or..), and if it does, there will be ample opportunity to convert before you're left in the cold, unless you're stuck in a cave for a few decades, or are kryogenically frozen.. True, if all companies die, you may be able to write your own DNG converter (/reader), given the DNG documentation, but FWIW, NEF is also thoroughly documented (document is in freely downloadable SDK), and reverse engineerable (that's what Adobe and Phil Harvey does), so you could also write your own NEF converter, so not really a very compelling argument, not this year anyway - ask me again in a few decades..

                             

                            If filesize is a main draw, then be sure NOT to save the hi-rez previews in your DNGs, otherwise you've just lost most or all of the filesize advantage. Also, you can strip previews from some proprietary raws (e.g. NEFs), to reduce filesize.

                             

                            Personally, if DNG supported sidecars, I would be more likely to convert, because file-management/backup is only an issue if you "forget" about xmp sidecars (and I do not forget), and I'd rather have my xmp in separate ready-to-read xml text files, rather than embedded in binary files, but hey, that's me..

                             

                            PS - I always use/encourage-others-to-use DNG when distributing raws to others for use in Adobe software, so xmp is not separate and can not get lost.

                             

                            PPS - proprietary raws and dngs are both just glorified tiff files - raw data, previews, and metadata..

                             

                            Don't get me wrong: I am NOT anti-DNG, and in fact - just the opposite: I am PRO dng, it's just that I wouldn't (don't) convert my own raws to DNG at this point. If you find the reasons to convert now are sufficiently compelling, you have my complete support and cooperation, fwiw..

                             

                            Cheers,

                            Rob

                            • 12. Re: Converting RAW to DNG! What are the advantages / disadvantages?
                              Keith_Reeder Level 4

                              Feierwoon wrote:

                               

                              But I wanted to go further into the subject.

                              This isn't the forum for that.

                              • 13. Re: Converting RAW to DNG! What are the advantages / disadvantages?
                                Abambo Level 4

                                Rob Cole wrote:

                                 

                                Feierwoon wrote:

                                 

                                That wasn't the question... .

                                Based on the title of the thread and your initial post, it seems like your question had a lot of

                                 

                                "What are the advantages / disadvantages?"

                                 

                                in it, and only a little

                                 

                                "What happens at a technical level when converting"

                                there was a title (for sure formulated as a general question) - there was an "exposé des motifs" - and there was a final question. After rereading it is still simply obvious to me. But I know that not everyone (me included) reads all the time the complete question... But for sure: your answer gives a lot of infos for people sneaking around.

                                • 14. Re: Converting RAW to DNG! What are the advantages / disadvantages?
                                  Abambo Level 4

                                  Keith_Reeder wrote:

                                   

                                  Feierwoon wrote:

                                   

                                  But I wanted to go further into the subject.

                                  This isn't the forum for that.

                                  In what forum should this question be asked?

                                  • 15. Re: Converting RAW to DNG! What are the advantages / disadvantages?
                                    Abambo Level 4
                                    NEF is also thoroughly documented

                                    At the time, NEF posed a problem because of encryption.

                                    A prime example is focus points - presumably they're there, but once converted, focus point feature is no longer supported. The amount of work required to support converted focus point data is unknown (to me), but as far as I know, no software has ventured to do it. If you want focus points, don't convert.

                                    I would love Lightroom to support focus points. That would really be a nice add on. I do not really understand the difficulty of supporting those, except if you fall into patents...

                                    • 16. Re: Converting RAW to DNG! What are the advantages / disadvantages?
                                      areohbee Level 5

                                      You caught me - I fell for the title and didn't read the post closely enough .

                                       

                                      In my defense, I often read more thoroughly, but this general question comes up so often that I just skimmed this time - guilty as charged.. .

                                      • 17. Re: Converting RAW to DNG! What are the advantages / disadvantages?
                                        areohbee Level 5

                                        Probably the DNG forum would be best, since you're after low-level technical details about DNG format. Most Lr users (including me) don't really know the gory technical details. For example, there are integrity check codes (over what? which algorithm?), and issues of DNG spec version (what's the difference in 1.3 vs. 1.4..), ACR version compatibility (now those numbers are pretty mysterious unless you have detailed knowledge..).

                                         

                                        But honestly, I don't think such details are going to help you make a decision whether to convert or not - it's really the general stuff upon which you'll most likely base your decision: what do you give up, what do you gain..

                                         

                                        But nothing wrong with being curious, and maybe you do need the details to be sure what to do, even though most of us do not - different strokes..

                                         

                                        Enjoy,

                                        Rob

                                        • 18. Re: Converting RAW to DNG! What are the advantages / disadvantages?
                                          areohbee Level 5

                                          Feierwoon wrote:

                                           

                                          At the time, NEF posed a problem because of encryption.

                                          That was a looong time ago - Nikon learned their lesson, and hasn't encrypted for years..

                                           

                                          Feierwoon wrote:

                                           

                                          I would love Lightroom to support focus points. That would really be a nice add on. I do not really understand the difficulty of supporting those, except if you fall into patents...

                                          I doubt there are patent issues - Aperture and some others support focus points, but ONLY if raw is proprietary, not DNG.

                                           

                                          R