This is an ages old limitation of AE's masks. Clicking on the content of the mask or inbetween anchor points will select the entire mask, clicking on the anchor points modifies the shape. unfortunately all of this only makes some sense when the mask is deselected, so users have to use a ton of extra clicks and fly blind. You're really not necessarily doing anything wrong, it's just so clunky.
First use the pen tool to select and move all mask points. This will keep you from selecting the layer. Second use the Alt/Option and Ctrl/Cmnd key to make the selections. Make careful note of the cursor. The white or open arrow selects one or more points, the convert tool (sideways v) adjusts the bezier handles or tension points, and the closed selection tool (short black arrow) will grab and move a single point. If you use the white arrow you may select multiple points and move them around. It's pretty easy once you get the hang of it. Trying to edit path's suing the direct selection tool (v - big black arrow) is nothing but frustrating.
Check out this example of simple manual roto:
I used to get frustrated by the intricacies of mask selection and editing in AE, but once I got used to it, I found it to be much faster than any of Adobes other programs (AI and especially PS). I never use the pen tool to move vertices, since you have to be much more careful not to inadvertently add new points or start a whole new mask. Instead, I use the regular selection tool and just make sure to click on the "mask path" line item of the layer before editing. Not to be confused with clicking the keyframe "path" line item, unless of course my intention is to highlight all the vertices to transform the mask/shape path. As long as you are diligent in selecting the right line item in the layer hierarchy, you rarely run into the annoying problem of moving the layer inadvertantly. I believe you can also hold down option with the regular selection tool to convert a vertex to bezier or modify a bezier handle, just as can be done using the pen tool. AI has gotten more manageable over the years in terms of intuitive path editing, but sadly PS is still an incredible joke, with the absolute worst vector path interface known to man.