23 Replies Latest reply on Oct 29, 2015 2:20 PM by Medoomi

    Is there a 16-bit file format that will edit like a RAW file?

    Medoomi Level 1

      Can I export files from a RAW converter (and utilize the color conversion/tone mapping of the converter), and still be able to edit the files like a RAW file in another converter (let's say Lightroom)?

       

      I've tried exporting as a TIFF, tone mapped with a medium tone curve. The problem I'm encountering is that the highlight/shadow recovery slider doesn't function with the same efficiency as I'm used to when working on RAW files.

       

      Or perhaps this is an inherent limitation of working with non-linear 16-bit files? If so, would it help exporting the file as a linear TIFF (would the highlight/shadow recovery sliders work like they do with RAW files)?

       

      Thanks!

        • 1. Re: Is there a 16-bit file format that will edit like a RAW file?
          Tanuj Sandal Adobe Employee

          Hi,

           

          • Working with an image file is not going to work the same as working with RAW File.
          • Because RAW file do not contain image data , they are just the information collected by the camera sensor.
          • When you convert RAW to an image format ( like TIFF ) , this information gets interrupted, the raw info is very flexible at this stage. More flexible that even the 16 bit image.

           

          Regards,

          Tanuj

          1 person found this helpful
          • 2. Re: Is there a 16-bit file format that will edit like a RAW file?
            Medoomi Level 1

            Hi Tanuj,

            Thanks so much for your reply.

            Yes I understand that a processed RAW file will have the data de-bayered and it's colors tagged within a specific color space & adjusted etc.

            What I would like to know is if there is any 16-bit image format that I can use that would enable Lightroom to use its basic image adjustment tools so that they will work as well on an adjusted file as they do with a RAW file. At the moment, I am seeing significant limitations at least with the shadow recovery & highlight recovery tools.

            I still don't understand what would make this impossible. As far as I understand, DNG is based on a TIFF file format anyway... Would the file need to contain a completely linear tone curve?

            Does anyone have any suggestions what format might help? Or what workflow could enable me to continue editing a processed file in Lightroom?

            Thanks!

            • 3. Re: Is there a 16-bit file format that will edit like a RAW file?
              JimHess Adobe Community Professional & MVP

              Unfortunately, at this time, there isn't a file format like you are looking for. TIF images come close, and so do PSD files. But once you have saved to an image format you lose that flexibility that you have in the raw image. The key is to use Lightroom or Camera Raw to do as much work as possible before turning to an image editor such as Photoshop or Photoshop Elements. Lightroom does a pretty good job with TIF images, in my opinion. However, it still isn't the same as working with a raw file.

              • 4. Re: Is there a 16-bit file format that will edit like a RAW file?
                Medoomi Level 1

                Hi Jim,

                 

                Thanks so much for taking the time to reply. Would you mind at all commenting on my workflow as I'm currently banging my head about it:

                 

                1) I'm scanning digital film negatives using a DSLR (beginning with inverted colors)

                • Lightroom behaves poorly & un-intuitively when using it's sliders with inverted colors
                • I would prefer to accomplish a separate RGB inverted curve adjustments before re-importing into Lightroom & then using the basic editing tools

                 

                2) Currently I am importing converted non-linear TIFF files into Lightroom. They are already tone-mapped with a medium contrast curve & I wonder if this is what causes the failure with the shadow/highlight recovery sliders.

                • My objection to working with a linear TIFF in Lightroom at present is that (I think) I would need dramatically to change the standard settings to achieve a standard appearance with the files. eg. I thinking that a medium contrast tone curve on a linear tiff won't look like a medium contrast tone curve applied to a RAW file

                 

                3) Ideally, I would like to have my files arrive in Lightroom at a similar point as RAW files do so that I can accomplish adjustments in a similar way as I currently do with RAW files

                • Is there a way that Lightroom could recognise that an image file is starting from a linear curve & apply an appropriate corrective tone-curve?
                • Would saving the linear TIFF as a DNG with embedded linear TIFF trigger Lightroom to do this... Or does it all come down to applying an aggressive tone curve in Lightroom?

                 

                Any comments or suggestions would be greatly appreciated,

                Thanks!

                • 5. Re: Is there a 16-bit file format that will edit like a RAW file?
                  D Fosse Adobe Community Professional & MVP

                  In a rendered file (TIFF/PSD) you simply have a lot less data to work with, that's why the adjustments aren't as effective.

                   

                  The raw file contains all the data your sensor captured, the rendered file only a subset. A different rendering, a different subset.Processing throws away a lot of data.

                  • 6. Re: Is there a 16-bit file format that will edit like a RAW file?
                    deejjjaaaa Level 2

                    I 'd suggest to try something like RPP which can export to a tiff (16bit or 32bit) w/o applying WB, w/o applying any curve (means linear curve) and w/o applying any color transform...

                    • 7. Re: Is there a 16-bit file format that will edit like a RAW file?
                      deejjjaaaa Level 2

                      > I would prefer to accomplish a separate RGB inverted curve adjustments

                       

                      you can try vuescan, did you try it ? it can do what you want with raw files from the a camera (instead of from a scanner)

                      • 8. Re: Is there a 16-bit file format that will edit like a RAW file?
                        Medoomi Level 1

                        Hi D Fosse,

                         

                        Thanks for your reply and I think I understand what you're saying. Trouble is, I don't think the conversion to linear-TIFF loses that much data or affects the highlight/shadow recovery per-se. When I use a linear-TIFF, I am able to manage highlights and shadows adequately, but Lightroom begins editing the file with the linear tone curve of the TIFF & this is rather unmanageable & difficult to correct back to a normal tone curve.

                         

                        I don't think conversion to a linear image format creates the problem.

                         

                        To ask the question the other way: Is there any way to invert a RAW file before processing a RAW file?

                        • 9. Re: Is there a 16-bit file format that will edit like a RAW file?
                          deejjjaaaa Level 2

                          > To ask the question the other way: Is there any way to invert a RAW file before processing a RAW file?

                           

                          I already suggested you to use vuescan and output to DNG

                          • 10. Re: Is there a 16-bit file format that will edit like a RAW file?
                            Medoomi Level 1

                            Hi Deejjjaaaa,

                            Thanks for the input, RPP looks interesting & I'll check into whether this can accomplish the task.

                            And to answer your other questions, yes, I own Vuescan, ColorPerfect etc. & have used them, but I am finding limitations with each of them that I'm currently trying to get around.

                            Is there anything else that might help?

                            Thanks!

                            • 11. Re: Is there a 16-bit file format that will edit like a RAW file?
                              deejjjaaaa Level 2

                              > yes, I own Vuescan

                               

                              so what do you find wrong with VueScan for your purposes then ?

                              • 12. Re: Is there a 16-bit file format that will edit like a RAW file?
                                deejjjaaaa Level 2

                                here are the brief instructions from one of two people behind RPP currently about using vuescan with raws ( use translate dot google dot com, as the text is 50% in russian ) before using output DNG further in your pipeline in a different raw converter (RPP for example or ACR/LR) : Пересъемка негативов - sail2ithaki

                                • 13. Re: Is there a 16-bit file format that will edit like a RAW file?
                                  Medoomi Level 1

                                  Hi Deejjjaaaa, I'm looking for a rock-solid starting point, so I'm prioritizing the need for a transparent RAW converter. Sadly I'm seeing better accuracy with other RAW converters than Vuescan (even after calibration with an ITC profile).

                                   

                                  I'd like to use my present RAW conversion. I am just looking for a transitional file format where I can do the RGB inversion & allow final editing in Lightroom...

                                   

                                  Thanks also for your link (I just saw them). I'll check them out later (I have an appointment just now). I wonder if Vuescan outputs linear-DNG as input device referred, or output device referred (Linear DNG). If it's output device referred, it shouldn't be be much different from a linear-TIFF.

                                   

                                  Thanks for the input. I'm open to any other suggestions...

                                  • 14. Re: Is there a 16-bit file format that will edit like a RAW file?
                                    deejjjaaaa Level 2

                                    > linear-DNG as input device referred, or output device referred (Linear DNG).


                                    in general
                                    linear DNG may or may not not have WB and/or color transform and/or tone curve applied (as in simply baked into per sensel RGB demosaicked data) - that depends on the software... Adobe tools do not bake WB or color transform or some other things beyond demosaicking and raw data values normalization into linear DNG raws, some other software may be written in a way to output to linear DNG with those things done (regardless of that Adobe does not mean those things to be in linear DNG converted from a native raw), I do not have vuescan handy right now @ work, so can't tell you 100%, sorry - but you can simply try yourself later today...

                                     

                                     

                                     

                                    > I'd like to use my present RAW conversion. I am just looking for a transitional file format where I can do the RGB inversion & allow final editing in Lightroom...

                                     

                                     

                                     

                                    here is the line from the link that you shall use = "Если нужен просто инвертированный с вычтенной маской DNG, то перед нажатием на кнопку Scan во вкладке "Output" установите "Raw file", "Raw save film", "Raw DNG format"."

                                    • 15. Re: Is there a 16-bit file format that will edit like a RAW file?
                                      barnstormvfx Level 1

                                      I have a similar issue to the original poster, though 32bpc would be my preferred file format.

                                      When I bring a RAW image (in this case Sonay ARW) into ACR, I can see that I have a lot of exposure latitude. Adjusting the exposure slider allows me to see the very bright highlights of this reflection, for example:

                                      Capture_01.PNG capture_02.PNG

                                      A big blowup but just an example to show the detail. The 'white' of the wall, even though it looks close to the sun reflection on the lamp, is nowhere near as bright, and the camera is capturing that latitude.

                                       

                                      However, any and all methods of converting the image to a useable format by Adobe destroy this latitude, as there seem to be no 32bcc saving options out of ACR.

                                      Here's the result once I bring it into PS or AE and adjust the exposure.

                                      capture_03.PNG

                                      Now I could simply create a bunch of different 'exposures' in ACR, save out the individual images, and then combine them into a .HDR that I could easily open and manipulate with 32 bit precision in either After Effects or Photoshop... but why should I have to do this when the data is already there in the raw file? I'm going to check out RPP but it seems like a huge oversight on Adobe's part.

                                       

                                      And for to forestall the common forum response of "why are you trying to do this?" or "you shouldn't be attempting X", this is for a visual effects application where the 32 bit data is relevant.

                                       

                                      ~ L

                                      • 16. Re: Is there a 16-bit file format that will edit like a RAW file?
                                        Medoomi Level 1

                                        barnstormvfx

                                        I don't know how you accomplished the transform out of ACR, but it looks like it has a tone curve applied. If so, the exposure slider won't help you as the data will be compromised.

                                         

                                        Back to the original question:

                                        This might be one for MadManChan2000:

                                        Could I use the following workflow to construct a DNG raw file (a positive) from a DNG raw file of a negative:

                                        1. Convert the RAW to linear TIFF
                                        2. Invert the TIFF
                                        3. assemble the linear TIFF into DNG format, tagging the DNG with a neutral LUT & applying a tone curve to bring it into a recognizable color space

                                         

                                        I guess I'm wondering if it is possible to construct a scene-referred DNG file from a output device referred TIFF.

                                        Ultimately, I'd like to know whether a file like that will behave (approximately) like a RAW file in its tonal responsiveness (highlight/shadow recovery)... it would require quite a bit of work to try it out by trial & error...

                                        • 17. Re: Is there a 16-bit file format that will edit like a RAW file?
                                          deejjjaaaa Level 2

                                          > from a output device referred TIFF

                                           

                                          as noted .TIFF does not need to be "output device referred" at all - it can have RGB from camera's "color space"

                                          • 18. Re: Is there a 16-bit file format that will edit like a RAW file?
                                            barnstormvfx Level 1

                                            I guess my issue is, I don't see how to pass a RAW file into Photoshop or After Effects *without* being forced to run the image through ACR, and since ACR can't convert to a 32 bit format (or something that carries a high dynamic range like a 16 bit half-float exr), it will always be throwing away some of the data. I'm not changing any of the tone curve or basic settings, but by simply not doing anything I am still enforcing some sort of destructive color profile (even if its just the default). I realize that you Medoomi are trying to fit a wide exposure into a 16-bit linear image, so I won't try to hijack your thread, but I think we're actually talking about the same thing here. Any time you affect the data with a tone curve or correction using a RAW converter, you're choosing which data to keep and which to throw away. When you subsequently export to a TIFF, the data is already gone I think. I believe our interests are similar in that we both want to hold onto that data in some way. For me, the way to do that is to determine if there's any way to get ACR to output an actual 32bpc format like EXR or HDR (or even 32 bit tiff). As far as I can tell, it doesn't seem to be the case.

                                            • 19. Re: Is there a 16-bit file format that will edit like a RAW file?
                                              Medoomi Level 1

                                              thx deejjjaaaa,

                                              If I were to trust the RAW conversion at any point, it would be going RAW-->TIFF, so I would prefer that this be mapped to a color space... But afterward, given that it already exists in an known color space, I'm imagining that converting it back into DNG could be done with a LUT that was zeroed out (LUT might be the wrong word--the color profile/tone curve in the DNG file is what I had in mind... the color profile could perform no change & the tone curve could apply a medium contrast curve to the inverted RAW file).

                                              • 20. Re: Is there a 16-bit file format that will edit like a RAW file?
                                                barnstormvfx Level 1

                                                I think I just got my answer:
                                                http://www.magiclantern.fm/forum/index.php?topic=15689.0

                                                Here's the big takeaway from the forum, which actually refers to another Adobe forum:

                                                Our design of ACR/LR is to render images for display or print.  When starting with scene-referred images (e.g., raw), this means tone and color mapping the input linear light data (e.g., whether 12-bit integer or 32-bit float) to values suitable for reproduction (output-referred).  Since the resulting is always intended to be output-referred, the resulting images are always stored using 8-bit or 16-bit values.  We do not see any need to represent output-referred data using 32-bit values.

                                                (ACR/LR is not intended to be a "pass-through" system that can, say, take 32-bit image data as input and also emit 32-bit image data at the end.  It can be made to do so, though clumsily.)


                                                According to this forum, it looks like one could actually use DaVinci Resolve to successfully create high dynamic range EXRs from raw data and not lose any of the data from your originals.

                                                • 21. Re: Is there a 16-bit file format that will edit like a RAW file?
                                                  deejjjaaaa Level 2

                                                  I still don't get why you are not using vuescan to achieve what you want... instead trying to build a babylon tower ?

                                                  • 22. Re: Is there a 16-bit file format that will edit like a RAW file?
                                                    Medoomi Level 1

                                                    Just to update the above discussion... As far as I can understand from various tid-bits pulled from forums & sources, converting a RAW file --> TIFF --> DNG/RAW will sacrifice some data (eg debayering & the various conversions for color space mapping). However, the resultant DNG/RAW file will still represent a competent DNG file i.e. further manipulations on this RAW file will produce similar images to what would be produced by the original RAW file.

                                                     

                                                    What seems most meaningful to the question is that the output from the average camera sensor is linear, which is as much as saying that conversions from RAW --> TIFF --> RAW will be minor edits, especially in context of a normal work-flow.

                                                     

                                                    The main caveats that come to mind are these:

                                                    1) As far as I understand (I would love to be corrected) the linear output from Adobe ACR does not provide a true linear file (a RAW converter would be needed that provided true linear output).

                                                    2) The inversion step would also need to be checked to be sure it retained linearity (eg. I suspect the Photoshop curves tool would be a bad choice).