First nobody rotoscopes one frame at a time. Second, with something like a panning shot you would Stabilize Motion first. Stabilize Motion is not Warp Stabilizing. You stop the camera motion by tracking something in the shot that you want to hold still. The next step with a shot like the one you describe is to expand the size of the composition so that you can see the entire shot as it moves. There are little buttons in the composition settings so you can do this more efficiently.
Then you add a new layer to the comp and use a blend mode so you can see through the shot and you start at frame 1 and draw a mask to separate remove the background. Then you go to the end of the shot and make any adjustments you need to make to the mask. Then you split the time difference and make further adjustments to the mask. If it is a smooth pan then it usually only takes a few keyframes to get the matte to follow perfectly.
Now set the matte layer as a track matte for the footage and add your new background. If the matte line is far enough away from the camera so you don't have to worry about parallax as the shot moves you've just about done with the composite.
The last step is to remove the Stabilize Motion from the original shot and add the Stabilize motion to the root layer and the background plate. The easiest way to do this if you have not added scale and rotation to the stabilize motion is to simply add a null to the comp. Then reveal the Stabilized layer's anchor point property which will have a bunch of keyframes. Alt/option click on the null's position property and use the pickwhip to connect the nulls position to the stabilized footage's anchor point. Now move the CTI to the first frame and then parent the matte layer, the background layer, and the stabilized footage to the null. This will line everything back up. Resize the comp or just nest it in a new comp that matches the size and frame rate of the original footage and you're done.
That's about it. Basic manual rotoscoping and exactly what would be done with the example you have shown.
If the background replacement area is closer to the camera then you would Camera Track the shot, set an origin and ground plane, then add a camera an a solid to the shot. You would do the same roto technique using the 3D solid and your background plate would also be a 3D layer. Pretty simple.
Here are two quick examples. The first shows basic roto but doesn't use the null trick to add in the motion. The second shows how to use Camera Tracking to add a matte for a layer. You'll get the idea.
Sorry, no audio on second example.
Wow, thank you Mr Gerard for this very detailed and comprehensive overview and reply. I appreciate it!!
For future reference, is there a special name given to this procedure/technique in the VFX industry? For example, if a VFX supervisor is giving instructions to a VFX artist, Burt, would he simply say - "Hey Burt, take this footage and use the 'rotoscope background removal procedure' to remove those hills and objects in the background so we can add a new background in there" ?
A visual effects supervisor would have reference shots with video or stills and they would coordinate with the director and director of photography and artistic director to plan for the best way to replace the background. The blocking and the camera move would be planned with the replacement in mind. The focal distance from the camera to the matte line would be calculated if there were to be ambitious camera moves. All of these elements would go into designing the shot, the matte and the background plate. Careful consideration for the ease of creating the matte would be given to the whole shot.
Then the effects house assigned to the shot would be given all of the notes from the shoot and they would assign a roto artist to generate the matte. Doing this kind of thing without careful preproduction planning and some sketches takes way longer than the time it takes to make a plan and be careful.
In your example the green screen was probably put there to allow them to have some action going on in that area so they could simplify the roto. The setup may look something like this before the key is pulled.
Simply amazing and thanks for illustrating this so clearly with your explanation and screenshot example. Teachers like yourself are highly respected. You are an asset to the industry.
Happy Easter kind sir!