17 Replies Latest reply on Oct 22, 2007 12:10 PM by (walter_schroeder)

    Compressed Format OK?

      Are there any disadvantages to saving my file in RAW Compressed format?
        • 1. Re: Compressed Format OK?
          ArrrBee Level 1
          It should be lossless so if that is the case the only disadvantage is you
          will have to take more pictures to use up all that space on your hard drives
          that those images aren't taking up. :)

          Robert
          • 2. Re: Compressed Format OK?
            Panoholic Level 2
            > Are there any disadvantages to saving my file in RAW Compressed format?

            Following is meaningful only if "saving my file" means "creating the raw file in-camera". If you refer to converting the original raw file in a DNG file, that should be compressed, as the compression is lossless and quite effective.

            It depends on the camera. The compression of the Nikon D70, D80, D100, D200, D2x, D2H (what else?) raw data is lossy. Nikon stated repeatedly that the loss is visually imperceivable, but this depends on the actual post processing.

            The compression of the Nikon D300 and D3 is lossless (or one can choose between lossy and lossless?).

            Note, that the "uncompressed" Nikon raw files are actually losslessly compressed, but the compression is quite primitive.

            I don't know what other cameras offer the choice between compressed and uncompressed raw formats.
            • 3. Re: Compressed Format OK?
              Bill_Janes Level 2
              >It depends on the camera. The compression of the Nikon D70, D80, D100, D200, D2x, D2H (what else?) raw data is lossy. Nikon stated repeatedly that the loss is visually imperceivable, but this depends on the actual post processing.

              This link explains the lossy compression Nikon uses for their raw files. It does throw away quite a bit of data, saving only 683 of the 4096 levels present in the original file. At first glance, this seems like a severe loss of data, but it is nearly always visually lossless. Dynamic range is not reduced, but redundant levels are discarded.

              As has been explained many times, the brightest f/stop of a raw file contains 2048 of the 4096 levels present in the raw file. According to the Weber-Fechner law the eye is sensitive to differences in luminance rather than the absolute luminance value (logarithmic response), and can detect a 1% difference in luminance; this works out to represent 70 levels in 1 f/stop (1.01^70 = 2.0). The 2048 levels are redundant and most of them are wasted, since they are beyond the resolution of human perception.

              Theoretically, this lossy compression could cause problems with highlight recovery or when some strong tone curve is applied, but I have never found this to be a problem, and challenge someone to demonstrate otherwise. If storage space is at all a problem, most Nikonians use compressed NEFs.

              For HDR images, integer encoding wastes too much space and log or floating point encoding is used as explained by Greg Ward.
              • 4. Re: Compressed Format OK?
                Level 1
                I should have been more clear -

                I'm asking about using *Camera Raw's* options to save the DNG as a compressed (lossless) file, (not doing it in-camera).

                Any disadvantages to doing that?
                • 5. Re: Compressed Format OK?
                  ArrrBee Level 1
                  Oooo Nikon is lossy. That just isn't right. Even if you can't see it that is
                  still very precious data loss from your images. Dynamic range and image data
                  is so precious when compressed using lossless techniques but to purposfully
                  throw away any data is just wrong. I won't be buying a Nikon anytime soon.

                  Robert
                  • 6. Re: Compressed Format OK?
                    ArrrBee Level 1
                    Just because the human eye can't see the loss doesn't mean it isn't a
                    problem. There are lots of things that can be done to an image where having
                    that data couldn't make a visual difference at a later step in the post
                    processing workflow. If the loss of data wasn't important then many people
                    wouldn't have pressured Adobe to improve 16-bit support in Photoshop. ACR,
                    Lightroom and Photoshop would only do 8-bit images. This even when the human
                    can't visually see a difference between and 8-bit and 16-bit image. What's
                    important is the software you use CAN see this extra data.

                    Bad Nikon. I certainly wouldn't consider their cameras professional grade
                    now. Nothing like getting something similar to jpg compression with a raw
                    file. So much for the raw file being the raw data from the sensor.

                    Robert
                    • 7. Re: Compressed Format OK?
                      ArrrBee Level 1
                      Adobe only does lossless compression with DNG converter. My Pentax K10D can
                      save native (at the time of taking the picture) DNG files. But, they are
                      uncompressed. I run all of them through DNG Coverter to compress them which
                      cuts the file size by 1/3 to 1/2.

                      Robert
                      • 8. Re: Compressed Format OK?
                        Level 1
                        I too have the K10D and I shoot RAW DNG. Is there a way that ACR can save compressed without creating a *new* file? I have to keep going in and deleting the original RAW file.
                        • 9. Re: Compressed Format OK?
                          Bill_Janes Level 2
                          >Oooo Nikon is lossy. That just isn't right. Even if you can't see it that is still very precious data loss from your images. Dynamic range and image data is so precious when compressed using lossless techniques but to purposfully throw away any data is just wrong. I won't be buying a Nikon anytime soon.

                          Robert,

                          That is a totally irrational response. The higher end Nikons allow the compression to be turned off. On the D70 only compressed NEFs are available. As previously stated, dynamic range is not affected. The choice is up to the user. Personally, I use the compression 95% of the time. There is simply no reason to have 2048 levels in the brightest f/stop when the eye can perceive only about 70.
                          • 10. Re: Compressed Format OK?
                            Panoholic Level 2
                            > Bad Nikon. I certainly wouldn't consider their cameras professional grade now

                            (Deleted, for Bill posted virtually the same as I did but sooner)
                            • 11. Re: Compressed Format OK?
                              Bill_Janes Level 2
                              >Just because the human eye can't see the loss doesn't mean it isn't a
                              problem. There are lots of things that can be done to an image where having that data couldn't make a visual difference at a later step in the post processing workflow. If the loss of data wasn't important then many people wouldn't have pressured Adobe to improve 16-bit support in Photoshop. ACR,Lightroom and Photoshop would only do 8-bit images. This even when the human can't visually see a difference between and 8-bit and 16-bit image.

                              Your comparison is not apt. A bit depth of 8 with gamma encoding is barely adequate to represent a full scale image without banding. Such images can't stand much editing without exhibiting banding, which is most prominent in the shadows where there are fewer levels. You might want to take a look at Norman Koren's levels graphic demonstration. The NEF compression throws away superfluous levels in the highlights and preserves the critical shadow data.

                              Also, 16 bits are desirable in wide color spaces such as ProPhotoRGB or else you might see color banding.

                              >What's important is the software you use CAN see this extra data.

                              I'm waiting for a demonstration of this. :)
                              • 12. Re: Compressed Format OK?
                                ArrrBee Level 1
                                The only compressed RAW file the K10D does is the Pentax PEF format. DNG is
                                uncompressed and Pentax doesn't seem to have any interest in fixing this
                                with a firmware update.

                                Robert
                                • 13. Re: Compressed Format OK?
                                  ArrrBee Level 1
                                  Sorry, I don't care of you can turn it off our not. A professional level
                                  camera has no business messing up the raw data for any reason. It totally
                                  defeats the purpose of raw and shows how trully clueless Nikon is if they
                                  think that even having the option is a good idea.

                                  Robert
                                  • 14. Re: Compressed Format OK?
                                    ArrrBee Level 1
                                    You are entitled to your opinion. RAW data is NOT supposed to be altered by
                                    the camera. Tossing away part of the data because Nikon is clueless is bad I
                                    don't care if you dress it up to the nines with a $10,000 handbag.

                                    Robert
                                    • 15. Re: Compressed Format OK?
                                      Level 1
                                      The K10D gives you the choice of shooting RAW in either PEF or DNG.
                                      • 16. Re: Compressed Format OK?
                                        Panoholic Level 2
                                        > A professional level camera has no business messing up the raw data for any reason

                                        Such categorical statements are not always useful; some conditions apply in most cases.

                                        The Nikon-kind lossy compression drops information in that range, which normally gets eliminated anyway. One has to realize, that a large number of raw shooters go raw because they want to adjust the white balance or make some other, minor adjustments. For that purpose, the lossy raw is adequate.

                                        The loss becomes noticable after extensive adjustments, for example bringing down the exposure in overexposed shots, or when more, extensive post-processing is required. I would say, as a "rule of thumb", when one would convert the raw in 16-bit TIFF, then one should start out with lossless raw data.

                                        I tried to create a demo for the advantage of 16-bit TIFF, and I needed to make quite "strong" adjustments in order to be able to show some difference.
                                        • 17. Re: Compressed Format OK?
                                          Level 1
                                          Robert - professionals like to have control of what they do. If you need a professional tool you can use say a D200 and use uncompressed files. There is nothing unprofessional about that. Taking choices away is non-pro - giving choices is professional. Besides 8bit 16bit or 12bit what does it have to do with professional anyway? Using the right bit resolution for the task is professional.