Short and to the point: Wrong tool. Use a tracking software that offers manual placement of tracking points like Syntheyes and use e.g. the edges of your table as reference.
AE point tracker will do the job, just needs a clear point that is always visible in the frame (not covered by the moving hands).
The proper tracking technique depends on the shot. Warp Stabilizer is for smoothing out the motion on a hand held shot where there is a fair amount of fixed geometry with sufficient detail in the shot for the stabilizer to work with. You can mask out certain parts of the shot to give the stabilizer something better to work with, then pre-compose and warp stabilize the pre-comp.
There are other times when you need to remove the camera movement completely and then apply some masking or other effects to the stabilized shot so that your workflow is significantly reduced. In this case you'll want to use something like a point tracker or a planer tracker. Mocha is a good planar tracker that comes with AE, and it also works great for certain shots, but for others you'll want to use AE's built in point tracker. For some kinds of shots a 3rd party solution like Syntheses is the best solution. It all depends on the shot.
Since we only have a general description of the shot and not much of an explanation of exactly what you are trying to do I tend to agree with Khkannisto that you may want to use AE's point tracker. Here's a shot that I recently did where I needed to put an overlay over a window with a hand held shot. Because of lens distortion and other things that were going on the easiest approach was to stabilize the shot for position, rotation and scale using AE's tracker, create my overlay, then put the motion back in the shot and add the camera motion to the overlay. This is how I set up the tracking:
I picked large areas of the car to track and set the track points at corners of the details of the body so I could easily correct the tracking. Once the shot was stabilized the overlay was added to the car and animated as needed to take care of the perspective changes. This only required about 5 keyframes. Then I added a null, named the footage "stabilized" and applied one of my animation presets to to the null. The CTI was returned to the first frame and the footage and the matte parented to the null. Now the added element perfectly tracks the camera movement. Here's the animation preset: Dropbox - destabilize Rotation Scale.ffx
This is the way the comp looks in the second stage of production. The shape layer overlay will eventually be replaced with a reflection of a face and the shape layer used as a track matte.
I chose the point tracker instead of Mocha because of all of the reflections and the scale and distortion that I had to animate in the overlay to get it to match the shot. I hope this helps. If you want concrete suggestions for the most efficient workflow then show us the shot and explain in detail what you want to accomplish.