frame blending (of timelapse) not working as expected

[ug_ffm]

Hi,

I have an image sequence of a timelapse I try to speed up.

The camera has made bursts of 9 frames every 15 minutes. In the end, I want to have only 1 frame for every 15 minutes.

Instead of discarding 8 of the frames, I thought I could blend them all together, to reduce noise and take advantage of some minor motion blur.

So I thought I could speed up the clip times 9.

(Aside: I sped up the layer to a target duration of 11.111%, and – since that is not totally precise – also experimented with interpreting it with a 9-times framerate (60 x 9 = 540).)

Both ways, when I go from frame to frame in the timeline, it shows every 9th frame, as expected.

But when I turn on "frame blending", the blending does not only happen on those desired 9 frames, but rather introduces traces of previous frames, too. (At least from frame #2 onwards.)

Does that have something to do with the comp's shutter speed or phase? (Experimenting with that did not seem to have any effect so far.)

Or is there any other way to fix this (i.e., to only blend those 9 frames together, without "spilling over")?

Thanks very much in advance!

[Rick Gerard]

If the various frame blending modes available through time remapping or time changes don't do it for you then you are kind of stuck unless you purchase come more sophisticated 3rd party tools. The latest versions of AE and Premiere Pro have several different options. Try them all.

[ug_ffm]

Hi Rick, thanks for your answer. But I am not looking for time remapping at all, and I don't just want any result, I also want to understand in detail which results frame blending gives for certain inputs.

(I believe there must be a way to controllably blend a certain number of frames into each other, instead of just a fuzzy "a certain number of frames" + n!

[Rick Gerard]

Time-remapping is as accurate as any other method and IMHO opinion does a better job of blending frames. You just have to do the math.

AE's built in frame blending has few options. When you blend multiple frames the choices are to use blending modes. Add adds up all the pixel values and makes the image brighter. Overlay basically adds everything above 50% gray. There are not many more choices. Unless you purchase some 3rd party solutions you can't take a bunch of footage of something like cars driving down a freeway at night and blend the frames together to get the kind of light streaks you would get if you held the shutter open for several seconds when you shot the time lapse.

Depending on how you originally set up the camera and how the camera is moving you might be better off automating a blend of the 9 frames using Lightroom or photoshop and then creating a new sequence. There is no way for AE to blend 9 frames at a time. You're going to end up with 4 frames from one burst and 5 from another at some point in time. Depending on the action in the shot this could look very odd. I am not sure that even Twixtor - RE:Vision Effects​ would be up to the task. I've been doing time-lapse for more than 40 years and I have never shot bursts of frames. I've done a bunch of time-lapse with long exposure times and I've changed the frame rate a lot of times through out the day sometimes starting with multiple second exposures at the start of the day and transitioning to short exposures many seconds apart throughout the day, ending up with long exposures at night. Once I traveled to the same location every 10 days and used a motion control rig to get a time lapse sequence of the sun rising, snow melting, wild flowers blooming, trees filling out, leaves changing color, falling and then snow falling and covering the ground as the camera panned from sunrise to sunset in Glacier National Park. Because every trip had a different length day I had to adjust the speed of the pan and the timing between frames so every sequence was the same number of frames. This project ran from mid March to the last week in November and by the time we were done we had a 90 second shot of the sun rising and setting while the seasons changed. Then we went to LA, built a set with the same topography as the foreground, got Bart the Bear to walk through the scene. A couple of months of rotoscoping and a composite and we had an amazing shot that national public television ran one time. The point of the story, time-lapse requires careful planning.

[ug_ffm]

Hi Rick,

thanks for the clarification. I was assuming that "time remapping" referred to interpolation only, and frame blending was simply frame blending. (Just to be clear: I was always referring to the little layer's switch in the comp, where you cycle through "frame sampling", "frame blending" and "interpolation".)

Thanks a lot for the tip with Lightroom, I'll try something like that.

Edit: Or I play around with the echo effect to blend 9 frames into one, and only after that keep just every 9th frame …

[Dave_LaRonde]

What's wrong with dragging every ninth frame into a new folder, importing the contents as a new image sequence, and being done with it?

Consider this extra work of pre-selecting your images as penance for the sin of not preparing properly.  And look on the bright side -- you'll never make this kind of mistake again.

[ug_ffm]

Hi Dave,

"manual" is out of the option because a) it is not elegant and b) I have way too many frames in that sequence – 580,924 frames in 3 sequences, to be precise.

Another: my client (who has not been seeking advice from anyone beforehand) has set up his own cameras a year ago (on a construction site) and has been shooting that timelapse until now, when they approached me.

Mulling over your suggestion just gave me an idea: maybe I can batch process (move or rename) the files into folders or sequences of 9, and have Media Encoder's auto-encoding do something with it … but assuming that I split my 3 cameras' footage into, say, each month they recorded (a bit short of 20,000 frames each – so just over 2,000 groups of 9), and have AME encode them, that would still leave the potential problems of importing them all, and then asking AE's keyframe assistant with (ideally) positioning over 2,000 layers (covering one month of shooting) in a comp. Unless I split them up, too …

So: I'm always glad to plan my shoots, but you don't always get the chance to.

As the job's payment is time-based, I'll very probably end up just discarding 8 of those 9 frames and be done with it, but I was wondering how AE's blending algorithm worked – a question that, despite Rick's answer above – still stands!

[Dave_LaRonde]

Ugh.  Yeah, that would be a boatload of work.  But if this 9-frame burst is consistent, the following might work:

• Import the sequence & interpret the sequence in the delivery frame rate
• Drag it into a comp.
• Change that comp's frame rate to 1/9 of the delivery frame rate -- its duration doesn't change.
• Nest this comp in another comp of the proper frame rate.
• Apply Time remapping to the nested comp, and speed it up 9 times.

AE should then see every ninth frame of the sequence at the correct frame rate.

[Kevin-Monahan]

Hi ugg_ffm

Are you still facing this problem? If not, let us know how you solved it. If so, please let us know so we can assist you further.

Thanks,

Kevin

[ug_ffm · Accepted solution]

Hi Kevin,

thanks for the follow-up.

I did a little research and made a test clip to see how "Frame Blending" works in both faster-than-original and slower-than-original:

In slow motion, Frame Blending fades from one source frame to the next over a couple of target frames – as one would expect.

For timelapse, Frame Blending superimposes multiple frames into one, with source frames closer in time being more prominent in the target frame than source frames more distant in time. (Writing this, I realize pictures would make that more clear …)

Also, the source time segment seems to be always significantly larger than necessary. So, for example, if you speed up to 9 times the original speed, Frame Blending – instead of taking 9 times the target time segment – takes a source segment 10 times or 12 times larger to blend frames together. (I did not measure exactly, because at that point I already knew I would not use Frame Blending.)

Unfortunately, this is not how a real exposure works: Every instant of time is represented with the same "weight" or prominence within the time range. (Best seen in the long motion blur streaks of light sources in nighttime long exposures – the lights show as streaks of a single brightness; not brighter in the timewise middle of their appearance.) So in a long exposure, which I basically aim to recreate, every frame of the source target rate should be equally visible/blended.

Also, when I speed up to 9 times the original, I expect the blending source time range to be 9 times the original – not more.

To get my desired "long exposure", I disabled Frame Blending (and set back to Frame Sampling), and applied the time "Echo" effect before precomping the clip and speeding it up:

Parameters for Echo effect:

Echo Time equals 1 divided by the number of frames per second of the source clip. (In my case: 1 / 60 [fps] = 0.01667.)

Number of Echoes equals the number of frames I want to cover minus one. (So for combining 9 frames, I want the current frame plus 8 echoes.)

Starting Intensity equals 1 divided by the Number of Echoes. (Which would be 1 / 8 = 0.125. Actually, I set the value slightly lower, to almost "1 divided by (Number of Echoes minus 1)", because otherwise I observed clipping of the bright whites in the result which I don't know where that came from.)

Decay must be 0, so that together with "Starting Intensity" every blended frame is represented with equal intensity, like in a long exposure.

I can't look it up right now and I don't remember the Blend Mode, but I seem to remember it to be "Blend" (maybe) or rather "Screen". (I checked by disabling and enabling the effect and see if the image retained the same brightness.)

After that, I precomposed the clip together with its effect and sped the resulting composition layer up to 9 times its original speed. I could fine-control wich result frame would be displayed by shifting the footage layer framewise in its precomp – because with this method, I only get one "correct" frame with only its same-burst neighbours blended into it, and 8 waste frames which have neighbours from other bursts blended into them.