Are you enabling keyframes by clicking the stop watch? Did you do some tracking? There is not enough of a description of your workflow for us to have any idea how your are working so it's very difficult to diagnose the problem. If you are a newbie then please start here: Basic Workflow
Wach all 3 videos.
I have anchor, position, scale, rotation, and opacity stop watches on, and no tracking turned on. Here's a screen shot of the heads that I had positioned correctly, but then going back to check, they moved on their own.
So did you track those faces or are all of those keyframes added by hand. If you used tracking then did you take any time to learn how to use the tracking tools? It's impossible to tell from your screenshot what is going on but I suspect that you're workflow is not correct. To add faces using tracking you shouldn't have to add keyframes to the Anchor Point. I would apply the tracking data to a null, see how the null follows the fact, then parent the replacement face to the null. EZ as that.
Well, your answer is here:
I have anchor, position, scale, rotation, and opacity stop watches on
You are making a mess by animating too much. Especially animating the anchor point makes no sense, if you are already animating position and rotation. Place it consistently and leave it fixed throughout the animation.