7 Replies Latest reply on Jul 17, 2013 5:54 AM by JJMack

    Content Aware Scaling on Video Layer



      I hope this is possible because it works really well on still frames but how would I go about applying content aware scaling to a video layer inside PS? Basically I'm trying to use this feature to fake an aspect ratio change where by I have a subject surrounded by a relatively uniform background and I would like to be able to create a protection mask as an alpha layer and have photoshop apply the content aware scaling only to the background(to fill in the empty parts of the rescaled video) and leave the subject untouched.


      What I've done is opened a video in photoshop as a layer, created an alpha channel to represent the subject(subject is white, background is black on the created alpha channel), changed the canvas size to the new aspect ratio which obviously created a tranparent frame around the original video frame. I then tried to apply content aware scaling to the video, it converted the layer to a smart object then displayed the free transform handles. However, when I selected my alpha channel as the protection area, the content aware scaling seemed to ignore it and went on to distort the entire frame. Now I tried this with still frame from the same video with the same mask/alpha channel and it worked as expected just didn't seem to do it with video.


      The idea, if it would be possible to do this in PS, is to rotoscope the subject roughly in AE and generate a matte pass which can then be used as the acompanying alpha layer of the video so then the content aware scaling can be applied to each invidividual frame.


      Is this possible??

        • 1. Re: Content Aware Scaling on Video Layer
          JJMack Most Valuable Participant

          I think what you might want to try is content aware fill where your filling the newly added transparent border. If that is possible.  Don't know how well it would work. For it sound like you don't want to scale the frame. That your actually trying to protect them with an alpha selection. I also think that content aware scaling each frame would produce a very jittery video.


          However I think the best way would be to just resize video the fill the aspect ratio area size you need, Crop to the required aspect ratio then resize. You will loose a  borders on the top and bottom or the sides when you change aspect ratios.  Something like you see on TV this film has been modified to fit your screen.  Or you could distort the video resize the video to the new aspect ratio size without constrain.

          • 2. Re: Content Aware Scaling on Video Layer
            icon-ap Level 1

            The entire point of using content aware filling/scaling is not having to resize and cope and loose details/resolution around the subject. Also I think with the right consistent alpha channel content aware scalling will be much more consistent than content aware fill because you're kind of telling PS what content to stretch as opposed to letting it guess how it should generate it. In either case I'll see if the fill actually works...didn't think to try it

            • 3. Re: Content Aware Scaling on Video Layer
              JJMack Most Valuable Participant

              icon-ap wrote:


              The entire point of using content aware filling/scaling is not having to resize and cope and loose details/resolution around the subject. 

              Filling and Scaling are separate thing.  Filling an area does not resize.  Scaling a document to some new pixel size is a resize.  When the resize is done with the content aware option is all about changing aspect ratio and controlling distortion somewhat.  Objects recognized by Photoshop and those masks by the users will not be distorted they will be resized in a constrained way and repositioned for the new aspect ratio and size. The remaining areas will be distorted to fit and fill the new aspect ratio and size.


              What you seem to be trying to do is mask the whole original Image/frames which is more or less asking Photoshop to resize everything is a constrained way,  That is like resizing an image  so it just fits onto a known paper size so it just fill the width or height which ever comes first. That leave white borders on the top and bottom or on the sides.  That is why I suggested suggested trying to use content aware fill. What you seem to want to do is have Photoshop fill the two empty borders in a content aware mode.

              • 4. Re: Content Aware Scaling on Video Layer
                icon-ap Level 1

                No that's not what I was trying to do, I mask the central subject of my frame and as I said it was surrounded by a fairly uniform background(main subject is a rock formation in the middle of the ocean) I want to change the aspect ratio of that frame, so I created an alpha mask to cover the main subject, the surrounding background was not covered in this alpha channel mask. This way when I scale using the content aware feature all that happens is the background(the water) is streteched evenly with lesser distorion(than normal scaling) while the masked main subject is "avoided" with completely no distrotion. I think the key point you missed in my description is that I am in fact creating an alpha matte for only my subject not the entire frame and using that in the "protect" option of content aware scaling.


                The result is essentially a frame aspect ratio change that does not sacrifice the resolution around the main subject(because the main subject remains of the same size and PS doesn't have to interpolate any pixels...this is obviously  is not the same as just resizng and cropping where the subject and background are streched evenly to fill an essentially larger area of the screen causing "softness"). The result using this method is also much more predictable from frame to frame than content aware filling because it's taking information from the current frame and using is to fill the empty background resulting from the aspect ratio change. I know this works because I tried it on individual still frames from the video then compiled them back in on AfterEffects...thats not my problem. My problem is when you try to natively try to do this inside PS on a video layer(rather than a still frame) it doesn't work and I'm not sure if this is a bug or if video content aware streching is just not supported.I have CS6 Production Premium.

                • 5. Re: Content Aware Scaling on Video Layer
                  JJMack Most Valuable Participant

                  I beleive content aware scale is more about not distorting know objects during resize.  If your changing the oveall  canvas pixel sizes as well as the aspect ratio everyting need to be interpolated even the protected area you mask.  Also if your subject is moving in the video how do you mask each frame.  How do you mask a moving object?.



                  If your just changing the canvas width or height to achieve the new aspect ratio. The interpolation of the protected areas will not be much.

                  • 6. Re: Content Aware Scaling on Video Layer
                    icon-ap Level 1

                    The method I describe works 100% on still frames extracted from the video, changing the canvas/aspect ratio is not the issue here because I do the same thing on a still frame and it works...just does not work on a video type layer. I appreciate you taking the time to respond but you're anwering the wrong question


                    As for masking a moving object, you create a track matte using standard rotoscoping techniques, it's very simple....in fact Mocha for AE or Pro generate by default BW mattes for tracked objects without any transfer mode/track matte tricks that you would have to do using masks in After Effects to get a BW or alpha matte.

                    • 7. Re: Content Aware Scaling on Video Layer
                      JJMack Most Valuable Participant

                      That explains it I do not have the AE add on for Photoshop.  AE cost to much for the amount of use I would have for it.  I also do not do video.  I think if you want to do video you would be better off using Adobe product Premiere Pro. I would think what your trying to do in Photoshop would be more a standard feature of  Premiere