10 Replies Latest reply on Mar 30, 2009 4:18 AM by Karel Bata

    "Merge to HDR" in AE?

    Karel Bata Level 1
      In Photoshop we've been able to 'Merge to HDR' for some time. You take several exposures like these: http://photos1.blogger.com/blogger2/4806/1533/1600/threephotos.jpg and blend them using tonemapping into one http://photos1.blogger.com/blogger/970/1071/1600/churchweb.jpg or like this http://bighugelabs.com/flickr/onblack.php?id=204476471&size=Large

      So is it at all possible to do this with moving images? Merging two or more video streams shot at different exposures, or taken from film and transferred at different lights?

      Here's some experiments I've done with still images taken from film negs and then digitised at different light settings http://gs93.photobucket.com/groups/l67/EZA6L48IV5/?albumview=grid (you'll need password 55555 )

      If this works for stills it should work for moving images. Any ideas..?
        • 1. Re: "Merge to HDR" in AE?
          Mylenium Most Valuable Participant
          I'm not aware of an easy way to do this in AE (Nuke and Fusion have tools for this), but it should be possible with blending mode operations and elaborately tweaked Levels and Exposure adjustments to mimic the highlight rolloff for each stop... The question is, though, if it would make sense for anything else than RED RAW and scanned film. All other footage types pretty much have already skewed Gamma and far too low bit-depth, so you'd only end up producing wrong HDRs from it. You know, 8bpc footage still will not yield any noticable improvements just by blowing it up...

          Mylenium
          • 2. Re: "Merge to HDR" in AE?
            Adolfo Rozenfeld Adobe Employee
            Karel: All uses I can imagine of HDR combined exposures with moving images are so experimental in nature, than I don't see a strong justification for developing Merge to HDR in AE. I believe the trend towards higher dynamic range sensors would probably get there in a few years anyway.
            Think about it. If you shoot two or three takes with different exposure values, everything in motion will produce blurring which would defeat the HDR merge process. I mean, I guess somebody could use this as an stylistic effect, but it wouldn't be proper a proper combined exposure, would it?
            I can think of an experimental procedure - aligning three cameras side-by-side and shooting with different exposures simultanoeusly. Then you could use auto-align in Photoshop to come up with a single combined frame which would perhaps have 60-70 per cent of information in common. If you're interested in this, maybe it's possible to come up with a script that would allow Photoshop to batch process the auto-align and HDR merge of video files previously converted to stills image sequences. Sounds like something worth exploring :)

            Unfortunately, there's another, probably deeper problem to consider which doesn't exist when you shoot stills with a tripod - just the fact that you're using different shutter speeds for the three simultaneous video takes goes against obtaining a successful combined image. It would again give you either ghosting or undesirable strobbing.
            • 3. Re: "Merge to HDR" in AE?
              Karel Bata Level 1
              Thanks guys! Sorry for my late reply. And also thanks Mylenium for your advice a few months back which was a real life saver!

              I'm not sure either of you really understand what I'm up to here. Part of the problem is that there are several flavors of HDR, and different people take the term to mean different things. I'm reffering spcifically to the kind of pictures (links above) that stills photgraphers have recently been taking.

              What they've done is take multiple exposures, and then tone map them together. For moving images that clearly presents difficulties - thought it should be possible with the sensitivity range and bit depth of the likes of a RED or Genesis. But how do you do it with a lesser (cheaper!) camera without having to repeat exactly everything several times?

              My thinking is that film already IS an HDR capture medium. The negative captures more data than is actually used in the final print, which is why you can print up or down to recover highlights or shadow detail (but not both without DI) so some detail is routinely lost (unless the subject brightness range is strictly kept within the stock's latitude).

              What I've done with some old stills is to use *the same negative* each time, and recovered detail previously lost in the highlights and shadows. That's the point - NO multiple camera exposures. I've done this using three different light settings on my desktop scanner, but it seems it will also work with just two. The final results, depending on the tone-map settings, vary from an increase in detail, to something that looks like old technicolor, to the truly surreal. Of course you wouldn't want to do this all the time, and skin tones tend to suffer from having all the detail (every little blemish) reproduced faithfully, but I still see plenty of applications!

              It's worth noting that this process will not *improve* an already good image, but it can bring to life one with an excessive brightness range - that two-shot of mine looks so much better than the original!

              All of this can already be done (presumably) in DI using a high-res scanner that will record *everything* on the negative, though I've yet to see anyone actually do it. Maybe the software isn't there, or it simply hasn't occurred to anyone! But I'm wondering if it can be done on the cheap, with a cheaper telecine, using two or three passes at different light settings and then blending them later. Doing a batch process in Photoshop or Photomatix would take *forever*!

              Any thoughts...?
              • 4. Re: "Merge to HDR" in AE?
                Adolfo Rozenfeld Adobe Employee
                Karel:
                Scanner software like VueScan automate this multiple exposure thing, using different exposure values for different scanning passes of the same image. Yes, it would still take a long time. But mostly because of the manual/mechanical aspects of using a general purpose still image scanner. Not to mention the need to align all the scanned frames so they're in register. Seems like batch-processing in PS is kind of difficult to avoid, at least to sync the resulting frames in register?
                • 5. Re: "Merge to HDR" in AE?
                  Karel Bata Level 1
                  Adolfo, I like the sound of VueScan, however at 25 frames for every second I don't think that using flatbed scanner is really practical. Unless there's some clever way of advancing it frame by frame. Would still take a *long time*.

                  I wonder what old telecine machines might still be knocking around and available on a budget?

                  As to keeping it in register - wouldn't stabilising in AE fix that?
                  • 6. Re: "Merge to HDR" in AE?
                    Mylenium Most Valuable Participant
                    >My thinking is that film already IS an HDR capture medium.

                    No, not really. In geeky computer terms, HDR means at least a 2^96 (96bpc) float precision with a value range of +/- 12, but at least +/- 3 or something like that (would have to look up the exact values). Plain film stock is 2^36 at best, and I think that only applies to high-quality 70mm, not even 35mm. A lot of the ability to recover highlights in film is more owed to the logarithmic nature the formats used and the inherent properties of the film stock itself. This is of course all debatable, especially with relation to the practical issues in a production workflow.

                    >But how do you do it with a lesser (cheaper!) camera without having to >repeat exactly everything several times?

                    Not at all. Since you cannot bypass the processing chips in the camera, there is no way to access the data of the sensor itself and all your output is already clamped, even if it may be 10bit. I also do not think you get that luxury with Genesis and RED. You just get 2 or 3 stops over/ under, but not full HDR ranges and still have to watch your aperture and exposure.

                    Mylenium
                    • 7. Re: "Merge to HDR" in AE?
                      Adolfo Rozenfeld Adobe Employee
                      Yes, using a general purpose still scanner for that would be incredibly slow. Not to mention how much worse it would be when you start taking multiple exposure to get every detail out of the negatives.
                      But... apparently your eye is richer than your budget, so for short sequences it may work...

                      Stabilize in AE: it would use 2-3-4 points. I guess it should work. Auto-align in PS would detect and use as many points as needed. And it can be batch-processed. Try the former, if it doesn't work well for your needs, you know you have the latter.
                      • 8. Re: "Merge to HDR" in AE?
                        Adolfo Rozenfeld Adobe Employee
                        > HDR means at least a 2^96 (96bpc) float precision with a value range of +/- 12, but at least +/- 3 or something like that (would have to look up the exact values).

                        Lutz, I think film is sort of High(er) Dynamic Range, in the sense that it will record information way past a value of 1.0. But you're completely right in that wouldn't have anywhere near the tone gradation of 32 bpc.
                        Don't agree? Go tell Brendan Bolles and all, not me :)
                        • 9. Re: "Merge to HDR" in AE?
                          yenaphe Level 4
                          A cheap solution would be to take your 8bit footage, duplicate as much as you need, add an exposure effect on each duplicata, and then play with levels, masks, and fusion mode to have an hdr look. There are some tips on some videocopilot tuts about faking HDR if i'm not mistaken. That could be a starting point.

                          (Of course, this works if you want something that feels like HDR, it will not be a HDR thingy)
                          • 10. Re: "Merge to HDR" in AE?
                            Karel Bata Level 1
                            Thanks Sebastian, but I'm aware of 'pseudo-HDR' and it's not what I'm after. There's an example here http://www.youtube.com/watch?v=-vCf1OU4B-c&fmt=18 and here http://www.youtube.com/watch?v=PiFLKAxrGZY&fmt=18 Though it's a strange choice of subject.

                            HDR means different things to different people. It was first coined in the 1940s by Charles Wyckoff, who took detailed pictures of nuclear explosions. Nowdays there seem to be several technical definitions. For instance, the guys who did the background plates for The Matrix call their techniques HDR.

                            If anyone's confused, I'm talking about the kind of stuff you get from several bracketed exposures. Like these three http://upload.wikimedia.org/wikipedia/en/thumb/6/64/HDR_example_-_exposure.jpeg/800px-HDR _example_-_exposure.jpeg yielding this http://upload.wikimedia.org/wikipedia/commons/thumb/5/51/Ball3_1_2.jpg/800px-Ball3_1_2.jp g Pushing the technique yields more surreal stuff http://www.multimediaphoto.com/gallery/luke/normal/sharel.jpg There's some nice animation around like http://vimeo.com/2636529 and http://www.youtube.com/watch?v=SGmQ_rz5lco&fmt=18 but I want to do it with live action.

                            I was dubious that I could actually do the little trick I've done with film, but whatever the math is, my tests with single film negatives have shown that IT WORKS! http://gs93.photobucket.com/groups/l67/EZA6L48IV5/?albumview=grid (you'll need password 55555 ) And it only takes one camera exposure! Of course exposures have to be absolutely spot on, and the stock ASA rating has to be accurate, but those things have come a long way in recent years. And of course the subject matter and brightness range needs to be appropriate - one with a low brightness range would yield no significant result. Now the question is: how to go from stills to moving? How to scan so many images at different levels (2 should do for starters) and then how to process them without it taking a year!

                            I'm also wondering if there's some particularly low contrast (high latitude) film stocks that might be useful, such as those normally used for inter-negatives, but which could be loaded into a motion camera.