You must not do anything wrong. Some stuff is simply far and beyond AE's 3D tracker. what you describe would be a scenario for a supervised/ calibrated track using respective manually placed markers and even then it may require manual corrections due to the limited optics in those drone cams. That's just the nature of the thing.
If you have a perspective change in the object you want to replace that changes the view of that object significantly then you have to replicate that perspective change in your replacement image. You said:
As the clip progressed, the "new" canopy elevated with the camera.
This makes me think that your aerial shot starts low so the Canopy is viewed from the front then as the shot moves up you can see the sides and top of the canopy. If that is the case you will have to create a more complex 3D model of the canopy so you can track the camera.
You could also mean that the canopy does not stick to the gazebo. This would mean that your track was good or that you picked the wrong points on the surface.
This is where screenshots would help. I'm thinking that the replacement roof does not stick to the gazebo.
One step that was left out in the good but not perfect tutorial was the first step of establishing a ground plane and origin. The second part of the tutorial that may give the wrong impression is that he didn't select enough points on the plane to accurately get the target. If you followed his technique of getting a target that was close to looking like it was lined up with the plane or surface you were trying to replace then rotated the solid so it looked like it matched the plane you were probably out of alignment. To line up a solid with a plane you should only rotate the Z axis. If you have to adjust X, Y, and Z then the tracking target is not lined up with the plane. You can get a better alignment by shift selecting points until you get the plane to match the target. Then after you create the plane you must check the track to make sure that it holds for the entire shot. If it does not hold or stick to the surface then you did not select the right racking points or you did not get a good track.
You also said this was arial footage. If it was from a quad copter with a GoPro or other action camera then the footage is probably distorted. This will also lead to problems when doing Camera Tracking in AE because AE's cameras have perfect lenses. The lens distortion will throw off tracking and cause extreme problems as the object gets closer to the edge of the frame. The solution to this problem is to remove the lens distortion. GoPro Studio has the option to remove fisheye and it does a fairly good job. My solution to this kind of problem is to export the GoPro footage uncorrected as an Image Sequence then use Lightroom to remove the lens distortion using either their lens profiles or lens profiles that I have created using a distortion chart.
So here are the steps for perfect object replacement using AE's camera tracker.
- Plan your shot with the limitations of perspective and parallax considered
- Carefully film the shot
- If needed shoot clean plates so you can remove objects from the scene
- Pick the shot you want to use and set your best guess for in and out points then add a few frames to each so you can trim the edits
- Export the footage so you can apply lens correction
- Camera track your corrected footage
- Set an origin and ground plane and make sure that it is good throughout the shot by adding a new 3D solid layer to the scene, rotating it 90º in X and seeing if it tracks with the scene (adding a grid to this solid will help)
- Pick tracking points on the surface you want to replace and add a solid with the Camera Tracker
- Check to make sure that solid tracks perfectly
- Rotate the replacement solid so that the horizontal or vertical edges line up with the surface you are trying to replace.
- Find the frame where the surface you are trying to replace is as big as it is going to be in the shot and export that frame to Photoshop for editing
- Do your editing, trim or crop the PSD so that only the surface visible or give yourself some room so you can roto the edges
- Replace the solid with the replacement surface in the comp
- Make any distortion corrections to the replacement surface that are necessary because of perspective changes or parallax using the Corner Pin tools
That's basically it if you are using a still image. If you need to do the surface replacement using another section of the video that is moving then you would use a projection map, but that's another subject.
Hope this helps. It sounds like when you resorted the camera track something had changed. If you got a successful track once then you should be able to do it again if you start from scratch with a comp that was created by selecting the clip in your Project panel and choosing New Comp From Selection. As I said before, if there is any significant lens distortion it's best to remove that before you start, then trim the shot, which can be done in the footage window, before you start.
If you want more help then post the shot or give us more details about exactly what you want to do.
Huge thanks, Rick. This makes sense now. Yes, my shot is one long perspective change.
This is a clip of the first few seconds:
I'll work on this when I have some spare time.
Here's how I set up the track and the target:
Once the points are selected you can move the target then hold down the Alt/Option key and resize it.
I then set Ground plane and origin then added a camera and a solid. I then created a 3D shape layer approximately the same shape and size as the roof, set the position and rotation to match the solid, did some masking and got this which tracked perfectly
Took about 5 minutes.
You = pro
me = n00b!
Super helpful! Thanks again.