5 Replies Latest reply on Aug 29, 2015 11:30 AM by PopV

    Object removal/masking in AE

    PopV Level 1

      Based on a suggestion, I'm bring this post over from Premiere Pro.


      The condensed version of my problem:


      I asked for best method to remove bird droppings from a gazebo roof in an aerial drone clip I shot. I was directed to this excellent 'chinfat' tutorial: After Effects CC 2014 Tutorial - 3D Camera Tracking and Object Removal - YouTube


      Here's my result: 

      Thanks J. Simon..it did and it didn't.


      The tutorial is very well done and seemed easy enough to follow. I sent the clip to AE, performed the "track camera" step as was done in the tutorial, created the solid and camera, exported a frame to PS, made corrections to the top canopy of the gazebo, sent the image to AE and merged it with the solid. Unfortunately, unlike his video where the window frame simply stays in place, my canopy would not. As the clip progressed, the "new" canopy elevated with the camera. I watched that video a dozen times or more and could never figure out what I was doing wrong.


      Frustrated, I started over from square one, but ran into a wall after the "track camera" step. Once the analysis was complete, an error popped up: 3D Camera Tracker: Layer size must match composition and use default transform values.  As a result, no track points were created. I ran into the same wall several times, then threw in the towel.


      For future reference I'd like to know what I was doing wrong (ie, why did the solid not stay in place) and what's up with that weird error message?



        • 1. Re: Object removal/masking in AE
          Mylenium Most Valuable Participant

          You must not do anything wrong. Some stuff is simply far and beyond AE's 3D tracker. what you describe would be a scenario for a supervised/ calibrated track using respective manually placed markers and even then it may require manual corrections due to the limited optics in those drone cams. That's just the nature of the thing.



          • 2. Re: Object removal/masking in AE
            Rick Gerard Adobe Community Professional & MVP

            If you have a perspective change in the object you want to replace that changes the view of that object significantly then you have to replicate that perspective change in your replacement image. You said:

            As the clip progressed, the "new" canopy elevated with the camera.

            This makes me think that your aerial shot starts low so the Canopy is viewed from the front then as the shot moves up you can see the sides and top of the canopy. If that is the case you will have to create a more complex 3D model of the canopy so you can track the camera.


            You could also mean that the canopy does not stick to the gazebo. This would mean that your track was good or that you picked the wrong points on the surface.


            This is where screenshots would help. I'm thinking that the replacement roof does not stick to the gazebo.


            One step that was left out in the good but not perfect tutorial was the first step of establishing a ground plane and origin. The second part of the tutorial that may give the wrong impression is that he didn't select enough points on the plane to accurately get the target. If you followed his technique of getting a target that was close to looking like it was lined up with the plane or surface you were trying to replace then rotated the solid so it looked like it matched the plane you were probably out of alignment. To line up a solid with a plane you should only rotate the Z axis. If you have to adjust X, Y, and Z then the tracking target is not lined up with the plane. You can get a better alignment by shift selecting points until you get the plane to match the target. Then after you create the plane you must check the track to make sure that it holds for the entire shot. If it does not hold or stick to the surface then you did not select the right racking points or you did not get a good track.


            You also said this was arial footage. If it was from a quad copter with a GoPro or other action camera then the footage is probably distorted. This will also lead to problems when doing Camera Tracking in AE because AE's cameras have perfect lenses. The lens distortion will throw off tracking and cause extreme problems as the object gets closer to the edge of the frame. The solution to this problem is to remove the lens distortion. GoPro Studio has the option to remove fisheye and it does a fairly good job. My solution to this kind of problem is to export the GoPro footage uncorrected as an Image Sequence then use Lightroom to remove the lens distortion using either their lens profiles or lens profiles that I have created using a distortion chart.


            So here are the steps for perfect object replacement using AE's camera tracker.

            1. Plan your shot with the limitations of perspective and parallax considered
            2. Carefully film the shot
            3. If needed shoot clean plates so you can remove objects from the scene
            4. Pick the shot you want to use and set your best guess for in and out points then add a few frames to each so you can trim the edits
            5. Export the footage so you can apply lens correction
            6. Camera track your corrected footage
            7. Set an origin and ground plane and make sure that it is good throughout the shot by adding a new 3D solid layer to the scene, rotating it 90º in X and seeing if it tracks with the scene (adding a grid to this solid will help)
            8. Pick tracking points on the surface you want to replace and add a solid with the Camera Tracker
            9. Check to make sure that solid tracks perfectly
            10. Rotate the replacement solid so that the horizontal or vertical edges line up with the surface you are trying to replace.
            11. Find the frame where the surface you are trying to replace is as big as it is going to be in the shot and export that frame to Photoshop for editing
            12. Do your editing, trim or crop the PSD so that only the surface visible or give yourself some room so you can roto the edges
            13. Replace the solid with the replacement surface in the comp
            14. Make any distortion corrections to the replacement surface that are necessary because of perspective changes or parallax using the Corner Pin tools

            That's basically it if you are using a still image. If you need to do the surface replacement using another section of the video that is moving then you would use a projection map, but that's another subject.


            Hope this helps. It sounds like when you resorted the camera track something had changed. If you got a successful track once then you should be able to do it again if you start from scratch with a comp that was created by selecting the clip in your Project panel and choosing New Comp From Selection. As I said before, if there is any significant lens distortion it's best to remove that before you start, then trim the shot, which can be done in the footage window, before you start.


            If you want more help then post the shot or give us more details about exactly what you want to do.

            • 3. Re: Object removal/masking in AE
              PopV Level 1

              Huge thanks, Rick. This makes sense now. Yes, my shot is one long perspective change.


              This is a clip of the first few seconds:



              I'll work on this when I have some spare time.


              Thanks again!

              • 4. Re: Object removal/masking in AE
                Rick Gerard Adobe Community Professional & MVP

                Here's how I set up the track and the target:

                Screen Shot 2015-08-29 at 10.20.15 AM.png

                Once the points are selected you can move the target then hold down the Alt/Option key and resize it.


                I then set Ground plane and origin then added a camera and a solid. I then created a 3D shape layer approximately the same shape and size as the roof, set the position and rotation to match the solid, did some masking and got this which tracked perfectly

                Screen Shot 2015-08-29 at 10.32.53 AM.png

                Took about 5 minutes.

                • 5. Re: Object removal/masking in AE
                  PopV Level 1

                  LOL...5 minutes.


                  You = pro


                  me = n00b!


                  Super helpful!  Thanks again.