• Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
    Dedicated community for Japanese speakers
  • 한국 커뮤니티
    Dedicated community for Korean speakers
Exit
0

'Super Resolution' Upscaling Algorithm

Community Expert ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

This was linked from a flickr group, and I am not sure how new the paper is, but it is very impressive. I didn't even try to read or follow the complicated bits, but it compares various up-scaling methods with what they are calling a Super Resolution method, and the difference is amazing.

http://www.wisdom.weizmann.ac.il/~vision/SingleImageSR.html

ISTR that it was a couple of Weizmann Institute guys that first demonstrated Content Aware Scale/Fill, so - if that is right - they probably have a relationship with Adobe.  Something for the next Adobe Max perhaps?

Views

22.6K

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe
Community Expert ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

Interesting.

As that paper is from 2008 I suppose it must have received some consideration at Adobe already.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

c.pfaffenbichler wrote:

Interesting.

As that paper is from 2008 I suppose it must have received some consideration at Adobe already.

Adobe may have see it however I just did a test using one of the example on the web page the eye chart. the origanal upper left corner on to one of Adobe biliner to the right Adobe Bicubic Sharper right of thatr CS6 Bicubic Automatic then their with a blue border.

res.jpg

Png next if it works

res.png

JJMack

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

Wow!!!

Thanks for posting!

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

Some nice marginal improvements, but I don't know if it's "wow" material...

Anyone who's pondered the mechanics of JPEG compression has probably thought of using a database of imagery to "make up" missing detail, and there are already good commercial upsampling tools that can make good 400% enlargements.  An OnOne Software plug-in called Perfect Resize 7 is based on technology that recognizes recurring fractal patterns in imagery, and has been a commercial product (in the prior name of Genuine Fractals) for quite a long time.

The results shown on the Weizmann page seem only marginally better in some cases and a bit worse (with more artifacts) in others compared to Perfect Resize results.

Example:  Super-Resolution result vs. the OnOne Software Perfect Resize plug-in + some USM:

http://www.wisdom.weizmann.ac.il/~vision/single_image_SR/kitchen/res.png

http://Noel.ProDigitalSoftware.com/ForumPosts/PR7.png

-Noel

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

Noel, your kitchen image is an interesting example of how the OnOne method produces an extremely synthetic result, similar to an image which has had some kind of "watercolour paint" or "paint daubs" or "palette knife" filter applied. The Super Resolution result is softer than the OnOne but looks more like a photograph and less like a deliberately filtered image, in my opinion.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

I was looking at the detail in the knitted material where what was little more than a blur with the other methods, clearly resolved each stitch with the SR method.  Yes there artefacts, but if you needed to use such a tool, your final medium would probably be a large print that would typically be viewed from far enough away to minimise the affect of those artefacts. 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

I hope nobody interpreted my stating that SR looked softer than OO with the kitchen scene enlargement as my finding fault with SR. The SR results are far sharper and detailed than the other methods and any slight loss in perceived sharpness when compared to OO is more than compensated for by the much more natural appearance of SR.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

I worry, though, with software that looks things up in databases and "fills in" the details with essentially false (or "best guess") information, where will this go...  Imagine a courtroom 50 years from now with a blown up security image clearly showing what looks like a great match to the defendant's face, when in fact the original security capture just didn't have enough information to convict...

You might pooh pooh that idea, but I have heard fractal enhancement is already used in courtrooms.

The additional complexity certainly seems to add some realism to the task at first glance.  But let's compare an image made by Bicubic upsampling then deconvolved (in this case with Maximum Entropy Deconvolution courtesy Astra Image), and combined with some parts from a fractal-upsampled image to reduce jaggies in key areas.  Which of these seems less plastic / made-up to you?

UpsamplingComparison2.jpg

-Noel

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

Welcome to the many reasons why this is still an active field of research.

None of the algorithms are great, but are improving.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

Argh, I spelled deconvolved wrong above.  [fixed now]

Lundberg, you're certainly critical.  Feel free to put up what you consider a good image to play with.

-Noel

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

Sure, the Super Resolution looks softer with less etched wrinkles in the face when the images are scaled down on the forum Web page, but when viewed full size, I find the jagged large-pixel appearance of the fractal plus deconvolution to be extremely unappealing. And you said the jaggedness has been reduced in the image on the left! How bad was it?

UpsamplingComparison2-detail.png

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

More texture in the skin, more jaggedness.  Less texture, smoother wrinkles.  For this subject, while both enlargements leave something to be desiired, I find the left image looks slightly more natural because it has more texture.

I reduced jaggedness around the edges.  The fractal resampling is good at smoothing rough edges.

This Super Resolution stuff has promise, no question, but it's not really groundbreaking if existing methods can get in the same ballpark.  It's a tool I'd love to have in my toolbox though. 

-Noel

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

More texture in the skin, more jaggedness.

The enhanced jaggedness in the left image isn't natural - it's regular like a pixel grid. That's the aspect which made may say that I find the image to be unappealing.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

Clearly it's just over your distraction threshold, and just under mine. 

-Noel

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

Here's what I'm talking about, although I'm surprised that the enhancement is necessary. Look at the orthogonal lines and huge stairsteps on the left.

jaggies.png

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

I knew what you were talking about.  Like I said, it didn't bother me.  I could probably do it better to avert more of that by combining the deconvolved data with the fractal data differentlyi.

-Noel

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

Noel Carboni wrote:

I worry, though, with software that looks things up in databases and "fills in" the details with essentially false (or "best guess") information, where will this go...  Imagine a courtroom 50 years from now with a blown up security image clearly showing what looks like a great match to the defendant's face, when in fact the original security capture just didn't have enough information to convict...

-Noel

50 years?  They do that now, and I know it's true because I have seen in TV progs like 24 and CSI. 

But being serious, it was surely not using look up tables when smoothing out the jaggies in the marble statue. Where bi-cubic was creating the 'affect' of straight lines by bluring them with aliasing, the SR has clean, sharp, high contrast lines.  I agree it was not so clear cut with skin tones, and I imagine that's the rub of the matter.  It kind of takes you back to magical debur demonstration at Adobe Max, or even third party applications like Topaz inFocus.  You see them demonstrated on best case subject material, which is not the same as real world subject matter.  I could never get the Topaz inFocus plugin to come even close to the demonstration on the Topaz site.  

As I have said before, I treat such things as black box tecnology and leave it up to clever folk like Chris to make it happen.  I am just happy to use them when they become reality.

[EDIT]  Not that anyone cares or noticed, but I have not been posting much recently because of a couple of big video projects that had to be done by this weekend.  Then I am overseas for a month, trying to keep clear of the Olymics in the UK. 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Enthusiast ,
Jul 25, 2012 Jul 25, 2012

Copy link to clipboard

Copied

Let's see, Noel is the guy who upsamples everything, apparently in the belief that it causes no problems. Did you upsample before doing your examples?  In regard to super resolution, what value would it have in practice except for recon photos or maybe astronomy? Certainly no portrait or publicity work.  What rescue function does it have that deconvolution doesn't do better?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 25, 2012 Jul 25, 2012

Copy link to clipboard

Copied

LOL Lundberg, why does what you write sound like a taunt?  You write, apparently in the belief that you know more than your audience.

Oh, and just so you know, some people actually DO work on astroimages...  This is a small crop from the very latest I've been working on, from a study of carbon stars...

NGC6871_Mosaic_Small_Crop.jpg

-Noel

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Enthusiast ,
Jul 25, 2012 Jul 25, 2012

Copy link to clipboard

Copied

I know you do work on astronomical images. Why would you want to use any form of artifact creating software on them? I realize that cosmology is part of the entertainment industry ever since Carl Sagan showed us billions and billions of billions, but I think they already see things that aren't there.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 25, 2012 Jul 25, 2012

Copy link to clipboard

Copied

LATEST

Actually I have no interest in using the software in question on astroimagery.  I do need better deconvolution software.  I think I'll have to write something myself.

Making more results from less data is why people would want a sophisticated upsampling tool such as this Super Resolution thing.  And this one is impressive, if only incrementally so.

-Noel

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Enthusiast ,
Jul 24, 2012 Jul 24, 2012

Copy link to clipboard

Copied

Since the kitchen scene is blown out, what difference does it make [message edited]?

Message was edited by: TerriStoneCHL

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines