24 Replies Latest reply on Jun 4, 2013 9:19 AM by Mathias Vejerslev

    Export to .exr / .hdr / floating point format from ACR?

    [ENH] KelSolaar

      Hi!

       

      Context is that I'm doing HDRI images for rendering so dynamic range is the most important aspect.

       

      I searched hard but didn't found a way to export from ACR into a floating point format which is very very very annoying because if you export as 16bit tiff it will obviously clip whatever values you have in your raw file. I can load the raw / dng files directly into Photomatix / Oloneo but then I'm losing any kind of Camera Profile / Distortion profile embedded with them.

       

      Oh and I don't want to use Photoshop to Merge the HDRI files because I want to have control over the deghosting and various other alignement related stuff. I'm really searching for a .dng / .raw to .exr conversion.

       

      Any ideas?

       

       

      KS

        • 1. Re: Export to .exr / .hdr / floating point format from ACR?
          ssprengel Adobe Community Professional & MVP

          Isn’t the data from a camera sensor less than 16-bits—usually 12 or maybe 14 from a Nikon D800?

          • 2. Re: Export to .exr / .hdr / floating point format from ACR?
            [ENH] KelSolaar Level 1

            Hi,

             

            Doesn't really matter, for instance let's say that you have shot a sky ( And you can confirm you have data using your exposure slider in ACR ), as soon as you export that to 16bit integer tiff, the data is clipped and lost.

             

            KS

            • 3. Re: Export to .exr / .hdr / floating point format from ACR?
              ssprengel Adobe Community Professional & MVP

              Is ACR histogram showing clipping at the bright end when you’re using ProPhotoRGB 16-bit?  Are you not wanting to use the Whites and Highlights sliders?

              • 4. Re: Export to .exr / .hdr / floating point format from ACR?
                [ENH] KelSolaar Level 1

                No I don't have any clipping and can retrieve the highlights, but I definitely don't want to do any exposure / recovery changes since those images will be merged together, they need to be kept as is.

                • 5. Re: Export to .exr / .hdr / floating point format from ACR?
                  Joe_Mulleta Level 1

                  I second this request.

                  I opened a similar thread here a while ago.

                  • 6. Re: Export to .exr / .hdr / floating point format from ACR?
                    Vit Novak Level 3

                    Raw data are 12 or 14 bit linear, but effectively it's not much over 10 bit because of noise, except maybe for medium-format. Saving that into 16 non-linear you are losing absolutely nothing - even interpolation errors in ACR are much bigger than that

                    • 7. Re: Export to .exr / .hdr / floating point format from ACR?
                      Joe_Mulleta Level 1

                      Not sure if this of any help for your problem, but are you aware of this post on Christian's Blog?

                      With those settings applied you can still use lens corrections and get more dynamic range from your merged HDR image.

                      • 8. Re: Export to .exr / .hdr / floating point format from ACR?
                        [ENH] KelSolaar Level 1

                        You are wrong: everything above 1.0 linear will get clipped once converted to a 8bit / 16bit integer file. Only way to keep it is by compressing the range using the exposure or recovery slider basically ensuring you have nothing above 1.0 linear.

                        • 9. Re: Export to .exr / .hdr / floating point format from ACR?
                          [ENH] KelSolaar Level 1

                          Hi Joe!

                           

                          Thanks for the link! I'm doing something like that atm but it's kind of tedious and you need to update depending the software you use the exposure exif field, otherwise the weight of the file will be wrong when merging. I'm currently scripting some stuff to generate and update those exposure automatically for me not perfect but it may do the trick as a temporary solution. I just hope Adobe listen to their users and provide that, shouldn't be that hard really.

                           

                          KS

                          • 10. Re: Export to .exr / .hdr / floating point format from ACR?
                            Vit Novak Level 3

                            [ENH] KelSolaar wrote:

                             

                            You are wrong: everything above 1.0 linear will get clipped once converted to a 8bit / 16bit integer file. Only way to keep it is by compressing the range using the exposure or recovery slider basically ensuring you have nothing above 1.0 linear.

                             

                            Well, I studied ACR workflow documented in dng sdk, and basicaly, there are several points in the workflow where colors can get clipped, so final procedure that saves it in integer doesn't change a thing

                             

                            ACR probably isn't the best tool to use with HDR programs because of curves used in the workflow mentioned  in above blog. In PV 2003/2010 it was possible to avoid usage of curves by setting brightness, blacks and tone curve to certain values, but in PV2012 there is additional compression that can't be turned of - even by setting sliders to values suggested in that blog. I suppose ti would be better to have unrendered input for HDR software, but it's not possible

                            • 11. Re: Export to .exr / .hdr / floating point format from ACR?
                              [ENH] KelSolaar Level 1

                              Cheers!

                               

                              That looks unfortunate and kind of a dead end then! I just hope they update that at some point in the future.

                               

                              KS

                              • 12. Re: Export to .exr / .hdr / floating point format from ACR?
                                ssprengel Adobe Community Professional & MVP

                                Will DxO tools work for you?  DxO creates linear DNGs after correcting the lens distortions.  Of course your lenses need to be supported and your HDR-input software needs to read linear DNGs:

                                 

                                http://www.dxo.com/us/photo/dxo_optics_pro/whats_new

                                1 person found this helpful
                                • 13. Re: Export to .exr / .hdr / floating point format from ACR?
                                  [ENH] KelSolaar Level 1

                                  I gave it a go, there are very good things yeah with it, it can write .dng files with undistortion, denoised, whatever basic operation I tried while maintaining the dynamic range which is very however one issue is that it doesn't support .dcp camera profiles ( Although you can dive into the .icc world if you want ) and the second issue is that it's frickin damn slow, to the point you would like to have a renderfarm :|

                                   

                                  Thanks for the suggestion anyway, much appreciated!

                                   

                                  KS

                                  • 14. Re: Export to .exr / .hdr / floating point format from ACR?
                                    MadManChan2000 Adobe Employee

                                    The point of using ACR/Lr is to tone-map your scene-referred input files.  This means making the input data (which can be very high dynamic range, e.g., 20 or 30 stops if it's been merged from multiple captures) suitable for reproduction on some type of output devices, such as a display or a print, which often has a much more limited dynamic range than the original scene or the original input file.  Once the input data has been suitably tone mapped (e.g., using the controls in the Basic panel, and optionally local adjustments as needed) for display/print, I don't see any reason to save the rendered output file in floating-point format. 

                                     

                                    Yes, > 1 values will be clipped away in an integer format such as 8-bit or 16-bit integer.  Which is why the images need to be adjusted/rendered/tone-mapped first -- so that you don't have 1.0 values except in specific areas that you want to have clipped values (e.g., deliberately clipped speculars).

                                    • 15. Re: Export to .exr / .hdr / floating point format from ACR?
                                      [ENH] KelSolaar Level 1

                                      Hi Eric!

                                       

                                      Good to have you on this thread actually I perfectly understand what have been the design goals with ACR, but your products are also used in the VFX industry as companion to various 3D applications where our needs in the main lines are similar but can slightly diverge in some areas. I have expressed my reasons in the first post, basically I want to be able to stitch my HDRI files with the maximum dynamic range possible because those are used for rendering in a 3D application. ACR is doing a much better job at rendering the raw files than any HDRI merging tool, provides more controls, color profiling, undistortion, noise removal, etc... Hope it makes sense.

                                       

                                      Cheers,

                                       

                                      KS

                                      • 16. Re: Export to .exr / .hdr / floating point format from ACR?
                                        MadManChan2000 Adobe Employee

                                        Ok now I understand better.  However, I don't understand yet how ACR would help.  As you know, at present, ACR does not have the ability to itself perform exposure merges.  Until it does, it seems to me that writing out to an exr format or other floating-point format would not provide any benefit ...

                                         

                                        It seems to me that what you're trying to do is get linear data out of ACR as a first step, then feed that linear data to a software that can do exposure merging, and then later do the tone mapping.  Is that correct?

                                         

                                        If so, you can indeed use ACR to do the first step, but the resulting data would have to be 16-bit integer (which would actually not be a problem in practice because your source data for a single raw capture wouldn't have any more DR than that).

                                        • 17. Re: Export to .exr / .hdr / floating point format from ACR?
                                          [ENH] KelSolaar Level 1

                                          I would benefit from first class raw demosaicing, noise reduction, undistortion, optional color profiling, etc... before sending that to merging in whatever software that is not the specialty to do what ACR does perfectly

                                           

                                          Getting linear data out would be the step after all the ones I described and I wouldn't change any exposure, recovery, black level, tone curve, etc ... related options so that the HDRI merge is predictable. That's where the problem resides.

                                           

                                          Lets say you have a photo with a bright sky, you load it in ACR, with everything to zero. Your sky is white and burnt ( But you actually have data from the raw file because you have checked it by changing the exposure compensation to -5 ) If you export the image as is without the exposure compensation, the sky data is clipped once casted to integer.

                                           

                                          If you export it with the exposure compensation, the HDRI merge will be very very bad because the software will have an image with the data where the exposure has been shifted but still exhibit the exif data from the original file.

                                           

                                          Of course you can merge with the tiff without the compensation and rely on a faster exposure to fill the missing data, it works but when you hit your latest fast exposure and you see that you would be able to pump even more dynamic range from it but that ACR clipped the data it's very frustrating.

                                           

                                          I now hope it's crystal clear and why I would like to get a linear exr output I can also upload a raw file somewhere for people having trouble understanding the issue.

                                           

                                          In the best case scenario Adobe will provide a solution, but if it happens it will not be before a while, in the worst case there won't be any solutions at all. With that in mind I wrote a Python script that generates for me the tail exposures for my bracketing sequences and modify accordingly the exif data.

                                           

                                          My workflow is the following for now:

                                           

                                          - Dump raw images from camera.

                                          - Apply neutral preset, noise reduction on all the raw files.

                                          - Praise Adobe for ACR.

                                          - Launch the Python script that:

                                               - Generates a subfolder where all the raw and xmp files are copied.

                                               - Images are renamed reordered by their exposure time.

                                               - The tail exposure is duplicated 3 times.

                                               - The xmp files are updated with exposure compensation ( -1, -3, -5 ). Now I get the dynamic range.

                                               - The raw files exif data is altered to reflect the exposure compensation.

                                          - Load the new raw files from the subfolder and convert them to tif.

                                          - Rant against Adobe

                                          - Merge to HDRI with my proper dynamic range.

                                           

                                          KS

                                          • 18. Re: Export to .exr / .hdr / floating point format from ACR?
                                            MadManChan2000 Adobe Employee

                                            The two reasons why ACR isn't going to be ideal for your situation are that ACR has two controls you can't disable (at least, not easily):  (1) white balance, and (2) the camera profile.  Ideally you would be doing exposure merging before these steps, especially because camera profiles can (deliberately) introduce non-linearities which messes up the merging.

                                            1 person found this helpful
                                            • 19. Re: Export to .exr / .hdr / floating point format from ACR?
                                              [ENH] KelSolaar Level 1

                                              Hi Eric,

                                               

                                              Very interesting actually, really glad you are monitoring that thread ,  I wasn't too much bothered by the white balance because it affects the whole file at image level but yeah the camera profiling is indeed something I thought about.

                                               

                                              What you would recommend as a workflow then?

                                               

                                              Is it possible to convert the .dcp profile files to a 3D Lut I would be able to apply in Nuke for instance? I don't really know what's in the .dcp files but I see them as some kind of x2 3D luts between which ACR interpolates depending of the image white balance.

                                               

                                              In the tests I have done ( I'm not searching for very precise color reproduction anyway ), profiling my raw files with a custom .dcp profile done with the ColorChecker Passport produced images that were "visually" closer to the tones in the shooting location ( Removing some warm cast and bringing more green, it was again really about feeling here ).

                                              • 20. Re: Export to .exr / .hdr / floating point format from ACR?
                                                Vit Novak Level 3

                                                I'm not sure if I understand your workflow correctly. Are you actually making fake "HDR" images from only one raw image, developped with several different exposure settings ?

                                                • 21. Re: Export to .exr / .hdr / floating point format from ACR?
                                                  [ENH] KelSolaar Level 1

                                                  No I'm just ensuring that I have the full dynamic range of my raw files once converted to 16bit integer tif.

                                                  • 22. Re: Export to .exr / .hdr / floating point format from ACR?
                                                    Vit Novak Level 3

                                                    From my remembering, in PV2012 workflow, if default settings are used, only color space conversion from Photo Pro to sRGB or AdobeRGB can cause out of range color values (whether > 1 or < 0), if color isn't already blown in raw image. So one of options would be saving tifs in Photo Pro, merging them in HDR software and doing the color space conversion later

                                                     

                                                    However, highlight are always compressed because all profiles (at least those supplied by Adobe) are using curves to compress scene dynamic range into dynamic range of output device. That could be solved by making and using a profile with linear tone curve. Only thing that you can't linearize is a part of whitebalancing algorithm that actually isn't documented in dng sdk and is additionally compressing highlights, to fit all colors into 0..1 range - unless you are using PV2003 or 2010

                                                    1 person found this helpful
                                                    • 23. Re: Export to .exr / .hdr / floating point format from ACR?
                                                      [ENH] KelSolaar Level 1

                                                      Hi Vit,

                                                       

                                                      Thanks for the insights, very helpful. I'm still evaluating a proper workflow for that and I think I should be able to get rid of ACR actually and have a proper floating point chain within Nuke, just need to check some stuff in my maths.

                                                       

                                                      Maybe you or Eric can help me there, in the SDK its stated "The forward matrix is required to map a unit vector to XYZ D50 by definition, so D can be computed by finding the neutral for the reference camera:" Does that mean I need to do any kind of operation on D / Camera Neutral / Reference Neutral here:

                                                       

                                                      ReferenceNeutral = Inverse (AB * CC) * CameraNeutral

                                                      D = Invert (AsDiagonalMatrix (ReferenceNeutral))

                                                      CameraToXYZ_D50 = FM * D * Inverse (AB * CC)

                                                      • 24. Re: Export to .exr / .hdr / floating point format from ACR?
                                                        Mathias Vejerslev Level 2

                                                        Hi Eric,

                                                         

                                                        > As you know, at present, ACR does not have the ability to itself perform exposure merges.  Until it does, it seems to me that writing out to an exr format or other floating-point format would not provide any benefit ...

                                                         

                                                        CR does have the ability to OPEN floating point files, though. Which is excellent. But sometimes I'd want to keep my image 32-bit. I cant both import and export in 32-bit. - Only import.