7 Replies Latest reply on Oct 16, 2015 12:21 PM by AngeloL

    Stabilizing using face track points...?

    AngeloL

      Hello Friends.

       

      Here is my project; I have time-lapsed people sitting still for an hour, so it is like a 30 second clip of straight-on a person's face. Obviously, the face is moving around a little bit. My objective is to stabalize the footage's position and rotation with respect to the eyes. So without any stabilization, the face moves around and rotates a little each frame, but I want to stabilize it so that the eyes are fixed.

           I am thinking it is a good idea to take advantage of the detailed face track points, but what I want to do is somehow, from the face tracking data, use the nose as a single anchor point to fix the position, and then use "left eye inner" and "right eye inner" as anchors to fix rotation (or use either the left or right eye as a position anchor, as well, I just want to avoid having one eye fixed with the other one rotating around it slightly).


      I am also thinking that, while using a tracker, i have to edit the targe to effecxt point control, but not sure how... please help! Thanks!

      help.png

        • 1. Re: Stabilizing using face track points...?
          Dave LaRonde Level 6

          I think your plan will work, but then the face will be still and the visisble parts of the body will move oddly.

          • 2. Re: Stabilizing using face track points...?
            Zulkifar2 Level 1

            Maybe you'll have some problem with lights and shadows, but...

            can't you use one frame only for the head and replace the nose+eyes+mouth part with the sequence?  Too extreme?

            • 3. Re: Stabilizing using face track points...?
              Rick Gerard Adobe Community Professional & MVP

              I would use AE's tracker to Stabilize motion and pick the inside corners of the eyes. Track Position, Scale and Rotation something like this:

              Screen Shot 2015-10-15 at 10.45.55 PM.png

              Make sure you pick enough of the4 eyes that you can get a good track. If they blink you may have to do some of the tracking by hand. Then just hit stabilize and the eyes should remain in exactly the same position. You may have to crop your images a bit or put them in a frame unless the image sequence is larger than your final comp size because the edges of the frame are going to move around a bit.

              • 4. Re: Stabilizing using face track points...?
                AngeloL Level 1

                Thank you for the response, Dave. I am just confused as to how to implement the plan- I know when I track with face points, there is a separate point for each facial feature. My question is how do I tell AE to stabalize using specific facial feature points, from the detailed face tracking points? (ie fixing the rotation with the eyes?)

                • 5. Re: Stabilizing using face track points...?
                  AngeloL Level 1

                  Hi Rick, thank you for the reply. I actually did do this, using the pupils as anchors, and while this works most of the time, several times the anchor drifted, as well as the feature finding box. I even did go through and manually place the spots where the subject blinked, or the anchors got lost completely. When I tracked using the detailed face tracking, it seemed to be much more accurate, as far as finding and maintaining the position of parts of the eye, and that's the reason I would like to try to utilize that. (I have 11 subjects, each with about 900 frames, so I have close to 10,000 frames I need to adjust, so I really need to make this process as efficient as possible!)

                  • 6. Re: Stabilizing using face track points...?
                    Rick Gerard Adobe Community Professional & MVP

                    The pupils are not going to work as well as the corners of the eyes. You need to pick points that are not going to move on the face and in most cases it would be the corners of the eyes. You also need to include as many pixels in the tracking area as you can. I always go for the easiest solution first.

                     

                    Using detailed face tracking will require some expressions to stabilize the footage and remove the movement. It may be easier to use Mocha to stabilize the face. I'd have to do some tests. I did do some tests using your image. To put in some movement I added a wiggle expression to position, rotation and scale using small values and very low frequencies. This gave me a face that randomly moved around the frame a bit. Then I pre-composed this shot and did some face tracking. This gives you a lot of information to choose from.

                    Screen Shot 2015-10-16 at 7.32.00 AM.png

                    Using simple expressions to tie the rotation, position and scale of the layer to the face tracking and face measurements I got a pretty decent stabilization but it was far from perfect. Trying to fine tune the expressions by averaging the position, scale and orientation of the face seemed to help and that may be a way to go if you are pretty good with expressions. This is what the fix the rotation expression looked like.

                    // rotation;

                    a = thisComp.layer("RtDetailedTrack").effectTrackPoints(Mask1)")("LeftEyeOuter");

                    b = thisComp.layer("RtDetailedTrack").effectTrackPoints(Mask1)")("RightEyeOuter");

                    v = b-a;

                    (radiansToDegrees(Math.atan2(v[1],v[0]) + transform.rotation

                     

                    I tried a simple two point scale rotation and position stabilize but I used the ears instead of the eyes.

                    Screen Shot 2015-10-16 at 7.08.21 AM.png

                    The result was perfect the first time. Once the shot was stabilized I added a null, made the null the parent of the stabilized shot, then positioned, scaled and rotated the shot to fill the frame. The easiest and fastest technique depends entirely on the shot. I would start this project using the shot that has the most movement and refine the technique there. Stabilizing in Mocha may be the most accurate way to go. I've got no time today but it may be an interesting experiment to see how well using the fact tracker could work to stabilize a shot when compared to this simple technique. Maybe I'll fool around with it this weekend.

                    • 7. Re: Stabilizing using face track points...?
                      AngeloL Level 1

                      Wow, Rick, thank you for such an in-depth answer. I am using those track points (the ears) and it is working out pretty well, good idea. But, I would like to still try using the face tracking, as some subjects have hair over one ear, and I will need to use other points. I am still very confused as to how to incorporate this, especially with regard to stabilizing:

                       

                      // rotation;

                      a = thisComp.layer("RtDetailedTrack").effectTrackPoints(Mask1)")("LeftEyeOuter");

                      b = thisComp.layer("RtDetailedTrack").effectTrackPoints(Mask1)")("RightEyeOuter");

                      v = b-a;

                      (radiansToDegrees(Math.atan2(v[1],v[0]) + transform.rotation


                      Could you take a screenshot of the timeline so I can see how that is placed? Super helpful, thank you!