Skip navigation
Trevor Dennis 5,972 posts
May 24, 2010
Currently Being Moderated

'Super Resolution' Upscaling Algorithm

Jul 24, 2012 2:31 AM

This was linked from a flickr group, and I am not sure how new the paper is, but it is very impressive. I didn't even try to read or follow the complicated bits, but it compares various up-scaling methods with what they are calling a Super Resolution method, and the difference is amazing.

 

http://www.wisdom.weizmann.ac.il/~vision/SingleImageSR.html

 

ISTR that it was a couple of Weizmann Institute guys that first demonstrated Content Aware Scale/Fill, so - if that is right - they probably have a relationship with Adobe.  Something for the next Adobe Max perhaps?

 
Replies
  • Currently Being Moderated
    Jul 24, 2012 3:30 AM   in reply to Trevor Dennis

    Interesting.

    As that paper is from 2008 I suppose it must have received some consideration at Adobe already.

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 24, 2012 5:55 AM   in reply to Trevor Dennis

    Wow!!!

     

    Thanks for posting!

     
    |
    Mark as:
  • JJMack
    6,051 posts
    Jan 9, 2006
    Currently Being Moderated
    Jul 24, 2012 9:03 AM   in reply to c.pfaffenbichler

    c.pfaffenbichler wrote:

     

    Interesting.

    As that paper is from 2008 I suppose it must have received some consideration at Adobe already.

    Adobe may have see it however I just did a test using one of the example on the web page the eye chart. the origanal upper left corner on to one of Adobe biliner to the right Adobe Bicubic Sharper right of thatr CS6 Bicubic Automatic then their with a blue border.

    res.jpg

    Png next if it works

    res.png

     
    |
    Mark as:
  • Noel Carboni
    23,534 posts
    Dec 23, 2006
    Currently Being Moderated
    Jul 24, 2012 7:17 AM   in reply to Trevor Dennis

    Some nice marginal improvements, but I don't know if it's "wow" material...

     

    Anyone who's pondered the mechanics of JPEG compression has probably thought of using a database of imagery to "make up" missing detail, and there are already good commercial upsampling tools that can make good 400% enlargements.  An OnOne Software plug-in called Perfect Resize 7 is based on technology that recognizes recurring fractal patterns in imagery, and has been a commercial product (in the prior name of Genuine Fractals) for quite a long time.

     

    The results shown on the Weizmann page seem only marginally better in some cases and a bit worse (with more artifacts) in others compared to Perfect Resize results.

     

    Example:  Super-Resolution result vs. the OnOne Software Perfect Resize plug-in + some USM:

     

    http://www.wisdom.weizmann.ac.il/~vision/single_image_SR/kitchen/res.png

     

    http://Noel.ProDigitalSoftware.com/ForumPosts/PR7.png

     

    -Noel

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 24, 2012 8:10 AM   in reply to Noel Carboni

    Noel, your kitchen image is an interesting example of how the OnOne method produces an extremely synthetic result, similar to an image which has had some kind of "watercolour paint" or "paint daubs" or "palette knife" filter applied. The Super Resolution result is softer than the OnOne but looks more like a photograph and less like a deliberately filtered image, in my opinion.

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 24, 2012 4:24 PM   in reply to Trevor Dennis

    I hope nobody interpreted my stating that SR looked softer than OO with the kitchen scene enlargement as my finding fault with SR. The SR results are far sharper and detailed than the other methods and any slight loss in perceived sharpness when compared to OO is more than compensated for by the much more natural appearance of SR.

     
    |
    Mark as:
  • Noel Carboni
    23,534 posts
    Dec 23, 2006
    Currently Being Moderated
    Jul 24, 2012 5:15 PM   in reply to Trevor Dennis

    I worry, though, with software that looks things up in databases and "fills in" the details with essentially false (or "best guess") information, where will this go...  Imagine a courtroom 50 years from now with a blown up security image clearly showing what looks like a great match to the defendant's face, when in fact the original security capture just didn't have enough information to convict...

     

    You might pooh pooh that idea, but I have heard fractal enhancement is already used in courtrooms.

     

    The additional complexity certainly seems to add some realism to the task at first glance.  But let's compare an image made by Bicubic upsampling then deconvolved (in this case with Maximum Entropy Deconvolution courtesy Astra Image), and combined with some parts from a fractal-upsampled image to reduce jaggies in key areas.  Which of these seems less plastic / made-up to you?

     

    UpsamplingComparison2.jpg

     

    -Noel

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 24, 2012 4:59 PM   in reply to Noel Carboni

    Welcome to the many reasons why this is still an active field of research.

    None of the algorithms are great, but are improving.

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 20, 2012 3:42 PM   in reply to Noel Carboni

    Since the kitchen scene is blown out, what difference does it make [message edited]?

     

    Message was edited by: TerriStoneCHL

     
    |
    Mark as:
  • Noel Carboni
    23,534 posts
    Dec 23, 2006
    Currently Being Moderated
    Jul 24, 2012 5:15 PM   in reply to Noel Carboni

    Argh, I spelled deconvolved wrong above.  [fixed now]

     

    Lundberg, you're certainly critical.  Feel free to put up what you consider a good image to play with.

     

    -Noel

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 24, 2012 5:32 PM   in reply to Noel Carboni

    Sure, the Super Resolution looks softer with less etched wrinkles in the face when the images are scaled down on the forum Web page, but when viewed full size, I find the jagged large-pixel appearance of the fractal plus deconvolution to be extremely unappealing. And you said the jaggedness has been reduced in the image on the left! How bad was it?

     

    UpsamplingComparison2-detail.png

     
    |
    Mark as:
  • Noel Carboni
    23,534 posts
    Dec 23, 2006
    Currently Being Moderated
    Jul 24, 2012 5:37 PM   in reply to conroy

    More texture in the skin, more jaggedness.  Less texture, smoother wrinkles.  For this subject, while both enlargements leave something to be desiired, I find the left image looks slightly more natural because it has more texture.

     

    I reduced jaggedness around the edges.  The fractal resampling is good at smoothing rough edges.

     

    This Super Resolution stuff has promise, no question, but it's not really groundbreaking if existing methods can get in the same ballpark.  It's a tool I'd love to have in my toolbox though. 

     

    -Noel

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 24, 2012 5:46 PM   in reply to Noel Carboni

    More texture in the skin, more jaggedness.

     

    The enhanced jaggedness in the left image isn't natural - it's regular like a pixel grid. That's the aspect which made may say that I find the image to be unappealing.

     
    |
    Mark as:
  • Noel Carboni
    23,534 posts
    Dec 23, 2006
    Currently Being Moderated
    Jul 24, 2012 6:01 PM   in reply to conroy

    Clearly it's just over your distraction threshold, and just under mine. 

     

    -Noel

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 24, 2012 6:04 PM   in reply to Noel Carboni

    Here's what I'm talking about, although I'm surprised that the enhancement is necessary. Look at the orthogonal lines and huge stairsteps on the left.

     

     

    jaggies.png

     
    |
    Mark as:
  • Noel Carboni
    23,534 posts
    Dec 23, 2006
    Currently Being Moderated
    Jul 24, 2012 6:12 PM   in reply to conroy

    I knew what you were talking about.  Like I said, it didn't bother me.  I could probably do it better to avert more of that by combining the deconvolved data with the fractal data differentlyi.

     

    -Noel

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 25, 2012 1:39 AM   in reply to Trevor Dennis

    Let's see, Noel is the guy who upsamples everything, apparently in the belief that it causes no problems. Did you upsample before doing your examples?  In regard to super resolution, what value would it have in practice except for recon photos or maybe astronomy? Certainly no portrait or publicity work.  What rescue function does it have that deconvolution doesn't do better?

     
    |
    Mark as:
  • Noel Carboni
    23,534 posts
    Dec 23, 2006
    Currently Being Moderated
    Jul 25, 2012 7:29 AM   in reply to Lundberg02

    LOL Lundberg, why does what you write sound like a taunt?  You write, apparently in the belief that you know more than your audience.

     

    Oh, and just so you know, some people actually DO work on astroimages...  This is a small crop from the very latest I've been working on, from a study of carbon stars...

     

    NGC6871_Mosaic_Small_Crop.jpg

     

    -Noel

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 25, 2012 9:05 PM   in reply to Noel Carboni

    I know you do work on astronomical images. Why would you want to use any form of artifact creating software on them? I realize that cosmology is part of the entertainment industry ever since Carl Sagan showed us billions and billions of billions, but I think they already see things that aren't there.

     
    |
    Mark as:
  • Noel Carboni
    23,534 posts
    Dec 23, 2006
    Currently Being Moderated
    Jul 25, 2012 9:15 PM   in reply to Lundberg02

    Actually I have no interest in using the software in question on astroimagery.  I do need better deconvolution software.  I think I'll have to write something myself.

     

    Making more results from less data is why people would want a sophisticated upsampling tool such as this Super Resolution thing.  And this one is impressive, if only incrementally so.

     

    -Noel

     
    |
    Mark as:

More Like This

  • Retrieving data ...

Bookmarked By (0)