4 Replies Latest reply on Mar 26, 2016 4:22 AM by trey38647

    Compositing technique to remove backgrounds in a moving shot


      Hello folks; I hope you are doing fine!


      I would like to talk about a compositing technique used to remove a background in order for new objects, e.g. matte paintings etc. to be put into the background.


      As you will see in the below picture, the background was removed in the first picture and replaced with a new sky, ground terrain and houses etc. in the second picture.


      This shot came from a moving shot where the camera was panning (moving) from left to right, therefore it was a moving shot.


      I note the first picture also has a small green screen. I believe that for budgeting and practical reasons, the entire background could not have green screens erected to cover the entire unwanted background in the shot.  So using a smaller screen in part of the area, I believe that some technique is therefore used to remove the background for the entire shot. I have seen VFX breakdowns where the same technique was used - i.e. only a small green screen is placed somewhere in the background when the intent is to remove the entire background.


      Let us say the shot had a slow and continuous camera pan move for 10 seconds. Then at 24fps we are looking at 240 frames. I know at some stage rotoscoping will be used but I find it difficult to believe that each of the 240 frames will be individually rotoscoped to remove the background. If this was the situation where every frame had to be rotoscoped, then I don't see the need for that green screen to have been erected in the first place.


      Therefore, I think there is something I am missing here regarding a missing link to the process. Is there some recognised effective/time efficient technique that is used to remove a background from a moving shot such as the below example, where only a smaller green screen is erected in part of the background instead of the entire background?


      Any tutorial you can point me to that helps with the process/technique in this specific case?


      Example link here:



        • 1. Re: Compositing technique to remove backgrounds in a moving shot
          Rick Gerard Adobe Community Professional & MVP

          First nobody rotoscopes one frame at a time. Second, with something like a panning shot you would Stabilize Motion first. Stabilize Motion is not Warp Stabilizing. You stop the camera motion by tracking something in the shot that you want to hold still. The next step with a shot like the one you describe is to expand the size of the composition so that you can see the entire shot as it moves. There are little buttons in the composition settings so you can do this more efficiently.


          Then you add a new layer to the comp and use a blend mode so you can see through the shot and you start at frame 1 and draw a mask to separate remove the background. Then you go to the end of the shot and make any adjustments you need to make to the mask. Then you split the time difference and make further adjustments to the mask. If it is a smooth pan then it usually only takes a few keyframes to get the matte to follow perfectly.


          Now set the matte layer as a track matte for the footage and add your new background. If the matte line is far enough away from the camera so you don't have to worry about parallax as the shot moves you've just about done with the composite.


          The last step is to remove the Stabilize Motion from the original shot and add the Stabilize motion to the root layer and the background plate. The easiest way to do this if you have not added scale and rotation to the stabilize motion is to simply add a null to the comp. Then reveal the Stabilized layer's anchor point property which will have a bunch of keyframes. Alt/option click on the null's position property and use the pickwhip to connect the nulls position to the stabilized footage's anchor point. Now move the CTI to the first frame and then parent the matte layer, the background layer, and the stabilized footage to the null. This will line everything back up. Resize the comp or just nest it in a new comp that matches the size and frame rate of the original footage and you're done.


          That's about it. Basic manual rotoscoping and exactly what would be done with the example you have shown.


          If the background replacement area is closer to the camera then you would Camera Track the shot, set an origin and ground plane, then add a camera an a solid to the shot. You would do the same roto technique using the 3D solid and your background plate would also be a 3D layer. Pretty simple.


          Here are two quick examples. The first shows basic roto but doesn't use the null trick to add in the motion. The second shows how to use Camera Tracking to add a matte for a layer. You'll get the idea.

          Sorry, no audio on second example.

          • 2. Re: Compositing technique to remove backgrounds in a moving shot
            trey38647 Level 1

            Wow, thank you Mr Gerard for this very detailed and comprehensive overview and reply. I appreciate it!!


            For future reference, is there a special name given to this procedure/technique in the VFX industry? For example, if a VFX supervisor is giving instructions to a VFX artist, Burt, would he simply say - "Hey Burt, take this footage and use the 'rotoscope background removal procedure' to remove those hills and objects in the background so we can add a new background in there" ?

            • 3. Re: Compositing technique to remove backgrounds in a moving shot
              Rick Gerard Adobe Community Professional & MVP

              A visual effects supervisor would have reference shots with video or stills and they would coordinate with the director and director of photography and artistic director to plan for the best way to replace the background. The blocking and the camera move would be planned with the replacement in mind. The focal distance from the camera to the matte line would be calculated if there were to be ambitious camera moves. All of these elements would go into designing the shot, the matte and the background plate. Careful consideration for the ease of creating the matte would be given to the whole shot.


              Then the effects house assigned to the shot would be given all of the notes from the shoot and they would assign a roto artist to generate the matte. Doing this kind of thing without careful preproduction planning and some sketches takes way longer than the time it takes to make a plan and be careful.


              In your example the green screen was probably put there to allow them to have some action going on in that area so they could simplify the roto. The setup may look something like this before the key is pulled.

              Screen Shot 2016-03-25 at 11.58.02 PM.png

              • 4. Re: Compositing technique to remove backgrounds in a moving shot
                trey38647 Level 1

                Simply amazing and thanks for illustrating this so clearly with your explanation and screenshot example. Teachers like yourself are highly respected. You are an asset to the industry.


                Happy Easter kind sir!