4 Replies Latest reply on Sep 24, 2016 1:35 PM by Roei Tzoref

    how would you remove this background?

    djmattyz Level 1

      how to remove shed.jpg

      i need to remove the white shed from the background in this film,  it has worked out in every shot except for this one because the actor is walking in front of the shed. ideally i would like to put some more bushes there, or perhaps just make it dark.


      animating a mask around the character frame by frame seems like an inneficcient way to do this, and would be prone to imperfections aswell.


      as a side note, this is shot with a DSLR on a gymbal, a 3d camera track didnt work but i managed to use a 1 point motion track, meaning that if i were to put items in there, they would stick to the scene fairly ok.


      any suggestions?

        • 1. Re: how would you remove this background?
          Roei Tzoref Adobe Community Professional
          Animating a mask around the character frame by frame seems like an inneficcient way to do this, and would be prone to imperfections aswell.

          That's right. you really should only rotoscope frame by frame when it's very fast movement where the mask discrepancies won't show.


          Rotoscoping is one of the toughest things to master in Ae or in Compositing in general, so if you are a beginner - you got your work cut out for you. the main issue here is you have to roto the part of the guy that's overlapping what you are about to insert. no more, no less. I am not sure exactly what you want to remove. is it just 1 or also 2? either way just mask what you need.

          In a nutshel: you create your patch (a replacement for what you want to remove) either in Ps or in Ae, you need to track the original footage and place in the right spot, and you have to roto the guy.


          so we got Patching, Tracking, and Roto


          Patching - this technique is also known as creating a clean plate. my workflow is to export a still frame to Ps, fix it, and bring it back to Ae.


          Tracking - if there is camera move, you need to track your shot and place the patch in place


          Roto - you have to Roto the part of the guy that's overlapping your patch.


          about masking techniques there are couple of techniques to choose from. since it's separating an image from a noisy background I would suggest examining these:

          1. Roto Brush: Roto Brush and Refine Matte in After Effects
          2. Rotoscoping mask with the use of keyframes, and tracking: Managing and animating shape paths and masks in After Effects + Tracking and stabilization motion workflows in After Effects. Rick Gerard has a cool tutorial demonstrating his technique for rotoscoping+tracking to make an effective roto: Simple Roto - YouTube
          3. Rotoscoping in Mocha: here's a very good author, I would suggest any of his tutorial - a 3 part series, here's the first part: Rotoscoping with Mocha for AE Part 1: Tools to Create and Export Shapes - YouTube . anything by the amazingly fast and effective Mary Poplin would be great too: mocha AE Creative Cloud Rotoscoping Demo with Mary Poplin at SIGGRAPH 2013 - YouTube
          • 2. Re: how would you remove this background?
            Rick Gerard Adobe Community Professional & MVP

            The first suggestion I would have would be for you to post a full size full resolution frame of your shot or better yet, the video of the shot so we could see exactly what kind of motion you were dealing with and what kind of edge detail.


            You have a few options, all of which involve tracking in one way or another and Rotoscope in one way or another. Here is how I would approach this shot and set up my comp.

            Screen Shot 2016-09-21 at 5.22.57 AM.png

            1. Figure out exactly what frames I think will be included in the final edit by editing the scene without the effects in the shot
            2. Create a new comp from my trimmed footage (layer 5)
            3. Use a suitable frame or frames from the shot to create a PSD that has the shed removed from a portion of the shot that does not have the actor walking in front of the shed
            4. Trim a copy of the original shot to just include the part of the shot where the actor walks in front of the shed (layer 4)
            5. Stabilize the copy of the shot so that the actor's head does not move (layer 4)
            6. Add a orange solid or shape layer and set the blend mode to Overlay to see through the layer to use as a track matte for the replacement background (layer 2)
            7. Add a null to the top of the comp that you can use to put the motion back into the copy of the shot you used for roto and add the motion to the roto layer if you used motion stabilize to steady the actors head
            8. Animate the track matte layer using as few keyframes as you can
            9. Insert your replacement background layer just below your track matte layer and set up the track matte
            10. Attach your background fix layer to the original shot through Camera Tracking or Motion Tracking
            11. Make any final adjustments and color correction you need
            12. Render s suitable DI (digital intermediate) using an appropriate mezzanine codec for your production workflow (I always use a 10 bit or better lossless format for this)

            There you go.


            Here are some notes: You can create the roto by hand as suggested, or maybe use Mocha. I do not believe that Rotobrush will work for this shot. If the actor's head stays pretty much in the same position it will probably work fairly well to Stabilize Motion and then use expressions attached to the null and parenting to fix up the shot. You may be better off using an advanced Mocha / Power Pin / Corner pin technique to do this. I don't know which will work better until I see your footage. We may be able to give you better suggestions if we knew exactly what the shot looked like. The most important part of my advice is to make sure that you only are working on the frames that need to be worked on. No sense wasting time on anything that is not needed.


            Just for fun - this is the advanced Mocha / Power Pin / Corner pin technique I was talking about that may work if you substitute tracking the TV for tracking your actors head.


            And here's a quick example that I did a few weeks ago using Stabilize Motion (don't be too critical of the tracking because I was not very careful getting a good track when I threw together the tutorial).

            • 3. Re: how would you remove this background?
              djmattyz Level 1


              thankyou guys, here is the footage as requested

              • 4. Re: how would you remove this background?
                Roei Tzoref Adobe Community Professional

                after examining your footage and trial and error, my workflow would be this:


                Patching - Creating a Clean Plate

                in short: export a still frame to Ps, fix it, and bring it back to Ae.


                the footage is very dark, I use the adjust exposure switch to see what I got here


                1. take a frame or two to photoshop that have enough features to complete a patch on the white shed.  create a marker to know which timecode this frame belongs so I don't lose it.


                2. in Photoshop I create a patch in this kind of setup: with stamp and masking I create a clean area


                3. I import this psd file as Footage and place it in the same timecode I took it from and also  mask it

                the reason I import as footage and mask in Ae is flexibilty. I can revise my corrections in Ps and save when I want and it will update in Ae. also I can mask and feather as I needed with flexibility in Ae.



                Tracking - Matching the movement

                in short: track the movement of the shed area in Mocha, bring the data back to Ae as a Null with transform data


                1. take my shot to mocha. I use 2 Tracking Layers:

                Layer 1 = the shed, some of the trees

                Layer 2 =  the guy and the hand


                in Mocha the upper layer is always the track matte so Layer 1 will be subtracted by Layer 2.


                these are my settings. I am not using perspective

                the white matte is the area that I am going to use eventually to get my tracking data from

                when the tracking is done and stable, I export to Ae


                2. export transform data to Ae.

                3. paste the data in AE on a Null, now I have a null that moves like the shed.


                Roto - Masking the guy

                in short: in Mocha track the guy's head and hand, create a bezier spline and link to the track, bring the data back to Ae as Mocha Shape, fine tune the edges using Refine Soft Matte Effect.


                I see I have two major movements: the hand, and the head . maybe a bit of the back and shoulder but that's easy. I decide to do also the roto in Mocha because here I can track the movement of the hand and link my spline to it, and correct my shape on a stabilized shot very quickly.


                1. Track the hand:

                2. create an accurate bezier spline around the hand and link it to the track


                3. adjust it a little with some keyframes while watching in stabilized mode:

                (same procedure for the head. due lack of time I can't do that right now.)


                4. export shape data

                5. paste on a duplicated video of the footage

                6. apply refine soft edge and tune it over the background

                this is how it looks so far before the head enters the frame. now you do the head!