15 Replies Latest reply on Mar 3, 2008 12:41 PM by JimHess-8IPblY

    New Workflow & DNG

    Spectric
      Hi all

      Well I got new camera & thought why not a new workflow ! I have been reading Mr Evenings & Scotts books on lightroom and Bruce Frasers / Jeffs book on Camera raw and want to improve efficiency and flow of data throughput.

      Initially I intend to save the Raw in say folder x, then using Lightroom import from this folder as DNG and save into the same folder. All processing & cataloging to be done on the DNG only in LR, the RAW is for peace of mind until I feel happy to just have the DNG's.

      So first attempt at using the DNG format, I start with a 26,169 Kbytes of .NEF and it becomes a 14,625 kBytes of .DNG ! What is no longer present ?

      I am also still confused about how Camera Raw 4 handles JPEGs & RAW files.

      I thought the RAW conversion of a JPEG is done in camera, so is CR4 just an editing tool for JPEGs, therefore how does it treat a Raw file differently, otherwise the advantages of Raw would be lost !

      thanks Roy
        • 1. Re: New Workflow & DNG
          Thomas Knoll Level 2
          Lossless compression. Maybe smaller embedded JPEG previews.
          • 2. Re: New Workflow & DNG
            Level 1
            >I thought the RAW conversion of a JPEG is done in camera, so is CR4 just an editing tool for JPEGs, therefore how does it treat a Raw file differently, otherwise the advantages of Raw would be lost !

            If your camera can ONLY produce JPEG, you can still use a tool with substantial functionality to do optimal editing of those JPEG files...Camera Raw 4.

            Yes, all of the tools WILL work on JPEG (and TIFF files) but no, it's not a magic bullet. Once a camera processes out a JPEG, it's in a gamma encoded color space which will severely limit the head room of highlight info in a raw file and limit the amount of white balance correction you can accomplish.

            Just because Camera Raw can now work on JPEG files doesn't mean you can forget about shooting raw. It just means that having shot JPEG you are no longer precluded from making use of Camera Raw's workflow and processing capabilities.

            If you shoot a raw + JPEG and open both in Camera Raw in film strip mode, you'll see just how much more control you'll have over the raw file vs the JPEG.

            But for those people who _DO_ need to shoot JPEG (sport, journalism or your camera doesn't output raws) you can still make use of the Camera Raw workflow...

            As Thomas has stated, when you choose the lossless compression, the odds are real good that the resulting DNG will be smaller than the original raw. Nothing is lost because the compression is lossless (meaning, no loss at all). Some DNGs may actually be larger (compressed NEF files for example may get larger), and the DNG will be bigger if you choose to embed the original raw file in the DNG.

            Go back and reread the material in RWCR CS3 where I wrote about DNG...
            • 3. Re: New Workflow & DNG
              Spectric Level 1
              Thanks for the replies, will definately reread the book. It is a great source of info that is always close to hand.

              thanks Roy
              • 4. Re: New Workflow & DNG
                Spectric Level 1
                Hi everyone

                Well i have been busy reading lot's and I think I know understand where my line of thought went astray. It is that CR4 can accept several file formats, not just RAW & this made me assume that CR4 reduced all input file types to a common format.

                In reality it is a common editor or set of tools that can manipulate one of several different file formats, but to maximise it's potential the input should be RAW data or a DNG. The term Camera RAW is in my opinion misleading in this context.

                Now the next question regards the data, if you convert to DNG then is the data file format the same irrelevant of the input file type, except that the output from a Raw file input needs Demosaicing and you may have more data to work with ?

                I now appreciate the fact that working in the linear Gama space offers advantages that are lost once you move into a Gama corrected workspace, but within the DNG format it is still seen as just data, same stuff as JPEG data just different values & or quantity !

                As for lossless compression, having read into the types of algorithms used I can now see how you can have a smaller file size that contains all the original data. It's just a case of storing all the color data values within the image and then locations of where they go, so instead of say 50 locations with 50 color X's, you have just color X and 50 locations. To confirm this I would assume that for a given image, the fewer the colors the smaller the compressed file size should be, so a single color shot should produce the smallest compressed file size.

                I think that once mastered Lightroom & using the DNG format could become the solution to my workflow, all contained within a single enviroment so keep up the good work on the Adobe products.

                thanks Roy
                • 5. Re: New Workflow & DNG
                  JimHess-8IPblY Level 3
                  According to Adobe, the image data in a DNG file is precisely the same image data that was in the original raw image. There are a few individuals will challenge that claim, but all of my experiences have proven that this is the case. The DNG file is a just new container for that image data. Under normal circumstances, the metadata that is part of the raw image along with the adjustments that you make in Camera Raw are stored within the DNG file rather than in a XMP sidecar file. When you edit a JPEG image in ACR it is the same concept. The changes made to the image are stored as XMP data within the file structure without modifying the image data. But all image data are not the same. Raw image data and DNG image data are still the raw data from the camera. The JPEG is still a JPEG image and suffers from the in-camera processing as well as the JPEG compression. But there still can be some benefit to editing the JPEG image in ACR.

                  I highly recommend that you purchase the book, "Real World Camera Raw with Photoshop CS3" originally written by Bruce Fraser, and updated for CS3 by Jeff Shewe. It is an excellent reference with numerous examples of how to make corrections, and there is a fairly comprehensive discussion of the DNG format in that book. I purchased my copy at the local Border's bookstore, and it is also available at Amazon.com.
                  • 6. Re: New Workflow & DNG
                    Spectric Level 1
                    Hi there

                    That is the book I am reading, great book. It has made me rethink my workflow even though I now use Lightroom, before i used & loved RSP. I really like the concept of the DNG, it makes so much sense to have a single RAW format.

                    What I aim to do is use LR as my main working enviroment for importing, editing & cataloging, but at the moment I am having issues getting my DNG's from LR into CS3 so that I can use CS3 actions for sharpening & back into LR.

                    They end up as Tif's not Dng in CS3 unless I open them directly from within CS3.

                    Having read about Photokit sharpener & Bruces take on doing it in three stages makes sense & who am I to argue ! So until LR can use CS3 actions I need to move DNG's between the two programs.

                    thanks Roy
                    • 7. Re: New Workflow & DNG
                      JimHess-8IPblY Level 3
                      Roy,

                      It is just a difference in the workflow of the two programs. In reality, you never edit a JPEG or a DNG or a TIF file in Photoshop. You edit the pixels that were contained in one of those files. When you look at the title bar of the image it is telling you the name of the file that the image came from. When you complete the editing of a raw file or a DNG file Photoshop you are given different file type choices based on the state of the image at the time. Photoshop knows which file types are available based on whether or not the image is 8-bit or 16-bit, whether or not the image has layers, etc.. But you cannot save the edited work as a Camera Raw file or to the same DNG file. You have to choose a new file. What Lightroom does is save that new image first and then instructs Photoshop to open the new image. So both programs are doing essentially the same thing. It's just that the new file is created at different stages of the workflow.
                      • 8. Re: New Workflow & DNG
                        Spectric Level 1
                        Hi Jim

                        Confused, The data in my Raw file is basically put into another container when I convert it to a DNG, so that the data is the same but in a different bucket.

                        In Lightroom all edits are non destructive, so the original file is not altered, just a set of editing commands are saved with the original data in the Dng file that instruct LR what to do. These commands are camera Raw 4 instructions, the same as in CS3 or so I believe.

                        If I export this DNG to CS3 then CS3 has the original data and any instructions from LR, so if I edit it in CS3 using say Photokit sharpener actions the changes will also be saved into the DNG !

                        So when I reopen it in LR I should have all the instructions on editing to date, or have I overlooked something ?

                        thanks Roy Oxlade
                        • 9. Re: New Workflow & DNG
                          JimHess-8IPblY Level 3
                          Roy,

                          No, that is not quite right. Once you have opened a DNG file in Photoshop you can not save any more changes to that DNG file unless you go back to Bridge. I understand that the title bar for the image says you are editing the DNG file, but that is only telling you where the image came from. And when you go to save that image it will be necessary for you to save the image to a new file. All of the changes that you have made are cumulative, including the changes made in ACR plus the changes that you made in Photoshop. But all that work has to be saved to a new file.

                          I only have Photoshop CS2 here at work, but the concept is the same. I just opened a DNG file, first in Bridge and then in Photoshop. When I go to save that work that I have done in Photoshop I am prompted to save a new file, and DNG is not one of the options. You cannot save the DNG file from Photoshop. You have to create a new file. One of the options is to save a Photoshop Raw file, but that is not the same thing as a Camera Raw file.

                          When you are working on a DNG file in ACR your changes are recorded as metadata within the DNG file. And if you then import that file into Lightroom those changes will be visible. But any additional changes that you make in Lightroom will be stored in the Lightroom database. And if you then try to open that DNG file back in ACR the additional changes that you made in Lightroom will not be there. That is why you have to export from Lightroom in order to include all of the changes.

                          Now, this is where the difference is. In ACR when you open the image in Photoshop it waits until you have finished your editing before it asks you to save your image. That is why you "think" you are editing the DNG file. But you cannot save a DNG file from Photoshop. You have to create a new file. What Lightroom does is create that new file first and then sends that new file to Photoshop. In either case you still are not editing the file. You are editing the pixels that Photoshop extracted from that file.

                          As I stated previously, the title bar of the image only tells you where the image came from. The image is no longer open. To prove this, I have opened an image in Photoshop before and then minimized Photoshop. Then I have been able to go to another program and open the same image again and have been allowed to save either one of the images.

                          Incidentally, all of the changes that you make to a DNG file in ACR are nondestructive. When ACR saves a DNG file it only writes the metadata. The image data remains untouched. But as soon as you open that image in Photoshop and make changes you have changed the image data. The image data is no longer in the raw format, plus it has been changed. So you wouldn't want to save the DNG anyway.
                          • 10. Re: New Workflow & DNG
                            Spectric Level 1
                            Hi there

                            Thanks for the detailed reply,

                            Ok, so as long as I am in Lightroom or ACR in CS3 then my edits or changes to the DNG will be saved in the DNG, which is ok but how can you apply capture sharpening to the RAW data in a DNG if you cannot use CS3 actions to run Photokit ?

                            It seems that Lightroom is lacking features that would make some task much easier, ie actions.

                            thanks Roy
                            • 11. Re: New Workflow & DNG
                              JimHess-8IPblY Level 3
                              The changes you make in ACR are stored in the DNG file. Changes made in Lightroom are stored in the database unless you use the option to write the changes as metadata.

                              As far as sharpening is concerned, the only sharpening that can be applied to the raw file or the DNG file is the sharpening that is available within ACR. It is not designed to be the complete sharpening workflow, but rather to provide you with capture sharpening. The rest of your sharpening, whether creative sharpening or for output, has to be done on the image after it has been sent to Photoshop.

                              Notice that the options for ACR allow you to just apply the sharpening to the preview image. This option is provided to help you see what the final image is going to look like without making any changes to the image. This is done because some people have developed their own sharpening workflow utilizing other tools, and they don't want ACR to interfere with what they have already developed.
                              • 12. Re: New Workflow & DNG
                                Photo_op8 Level 1
                                Jim/Roy-I found this thread while searching the forum with the SAME concerns as Roy. i.e, a way to apply capture sharpeneing to DNGs. My workflow seems to be similar to Roy in that I work with DNGs within Lightroom. I also have (use) Photokit Sharpener. Simply put, does anyone have the values that one would use in LR sharpening that would "approximate" Photokit's Digital High-Res Sharpen with Medium Edge Sharpen???

                                --Dave
                                • 13. Re: New Workflow & DNG
                                  JimHess-8IPblY Level 3
                                  All of the sharpening that is done within Lightroom is capture sharpening. That is all that can be done in Lightroom. All other sharpening must be done to the image after it has been opened in Photoshop. You should not try to duplicate the results of Photokit Sharpener. It isn't designed it done that way.
                                  • 14. Re: New Workflow & DNG
                                    Spectric Level 1
                                    Hi all

                                    Having spent a lot of time reading & rereading the many books in my collection I saw the light at the end of the tunnel !

                                    I went back to basics and rethought the workflow, file formats etc etc and things are so much clearer now.

                                    In simple terms the RAW is read only, always has been and is where my thoughts led me astray, so that now I can more fully appreciate how lightroom works & why DNG is the way to go.

                                    Having read the Adobe DNG Pdf and several articles on lightroom I now realise that the DNG is just a container holding my RAW data in an open format, XML meta data that contains a set of instructions as to how I wish to have the data processed into a gama corrected RGB color space and other info like keywords etc.

                                    So now I realise the benefit of DNG & that this one container holds everything in relation to my original RAW file & more.

                                    The odd thing is the fact that ACR accepts JPEG & Tif files !

                                    thanks Roy
                                    • 15. Re: New Workflow & DNG
                                      JimHess-8IPblY Level 3
                                      Roy,

                                      In the new version of, "Real World Camera Raw with Photoshop CS3" Jeff points out that, to a certain point, they are still wondering about the value of providing that extra capability. But since it was a feature of Lightroom, and Lightroom and ACR share a common "engine", it was decided to provide that feature. The book also points out that while the controls do have an effect on the JPEG and TIF images, the results will not be as dramatic or as effective as it is on raw files. The same is true in Lightroom.

                                      I really don't do justice to what Jeff has written in the book. All I can say is that it is a book that is well worth reading. It ought to be part of your reference Library. It really isn't that expensive. I had a 40% off coupon for Border's bookstore and the book only cost me about $28. But even at full price ($43 or thereabouts) it's well worth the money.