Can we use machine learning to improve video resolution in Premiere yet?

[koenahn]

Hey everyone,

My specific case is that I'm making an interview montage. The supplied video files were filmed at a pretty wide angle, showing the whole person. I just need to get them from the bust up, which means I'm doing some serious cropping. The result, as you guessed, is a somewhat pixelated face.

I want to enhance my video clip: I want to increase the resolution, make the image sharper. I know the people at letsenhance.io are doing it with still images, so I'm sure it can be done with video.

Technically feasible as it may be, I can't seem to find any documentation of this feature and I was wondering if it's just hiding in plain sight or if we're still waiting for implementation. Or maybe there's a plugin. I'd like to do this in Premiere, but I'm open to the idea of using different software as well.

Who can tell me if this can be done, and if yes, how? Thanks a bunch.

Kind regards,

Koen

[Jim_Simon]

Take a look at Detail Preserving Upscale in After Effects.

[chrisw44157881]

infognition has a super resolution AE plugin that compares adjacent frames to enlarge. although AE's preserve upscale tied with unsharpen mattes with ignore shadow detail works really good as well.

[koenahn]

That sounds great, I'm going to look into this. Thank you!

[koenahn]

Hey Chris,

Thanks again for helping me out. Could you clear something up for me?

Turns out the plugin is Windows-only, so I decided to go for the detail-preserving upscale. That part I have figured out.

However, I can't seem to find anything called "unsharpen mattes" or "ignore shadow detail". Could you point me to that? I have found "refine hard matte" and "refine soft matte", I'm not sure if you may have been referring to either of those. Thank you!

Kind regards,
Koen

[chrisw44157881]

in AE, use effect shift, set alpha as luminance, set that layer as alpha inverted matte with levels-alpha to increase alpha contrast for layer below with effect-unsharpen applied. and bottom should be original layer. (this is all off the top of my head) it should only apply unsharpen to the lighter colors, not the shadows(which can cause macroblocking)

[MyerPj]

You're doing a good job koenahn​, please do follow up on this, asking the next question... ie: I'm not sure where to start on that chrisw44157881​, like 'effect shift'...

[koenahn]

Heya Chris,

Thanks again for the explanation. I'm not very experienced with AE, so your effort is very helpful. Here's what I did so far:

• I used the "Shift Channels" effect to set "Take Alpha From" to "Luminance." I applied this to my original layer.
• I understand I need a second layer, so I made a black rectangle called "Shape Layer 1", above the original layer. I was able to set my original layer's TrkMat setting to "Alpha Inverted Matte, Shape Layer 1."
• Now I'm not sure what to do next. I just see the black rectangle.
• I think the next step you describe is to add a "Levels, individual controls" effect to the original layer and tweak the Alpha settings, but since I'm only seeing black at this point, we'll have to get back to that later.

My questions are:

1. What am I doing wrong with the Shape Layer and the matte setting? What kind of layer should I have above the original?
2. Am I applying the effects to the correct layers?
3. Where do I apply the "Unsharpen" effect?

Thank you for clearing this up, I appreciate it.

Kind regards,
Koen

[chrisw44157881]

in my AE project 3 strip,

un-hide the FINAL OUTPUT comp

look at the 'sharpen' and 'mask sharpen comp' to see how they're made.

https://drive.google.com/open?id=1BYIAn5Yiv2ejZjy2JBpjXpcMx8gksvPV