Without a 3rd party script you can't tie the vertices of a mask layer to position with an expression or any thing else.
I would have approached this problem with a shape layer, rather than a mask. If you used a rectangle and observed the position and size you could easily tie the dots to the corners. If you needed this shape layer to act as a mask for another layer you could use it as a track matte and just keep it visible.
Something like this:
Now just animate the size of the rectangle and the dots will stay in place. Use the layer's position to place the rectangle where it needs to be, set it as a track matte for the circle. Note that the Transform Rectangle Position is 0, 0
The bottom right and top left are just created with a pickwhip then the size divided by 2. You put a minus sign in front of one of the expressions. The top left and bottom right require you to separate the values in an array and put a minus sign in front of the x for one and the y for the other. Pretty simple stuff.
If you just want a rectangle, then there's a million alternate ways...
Rick, this is fantastic! Your screenshot and explanation made it easy to figure out. The track matte idea worked, but it made the ellipses part of the mask and I need them to sit above the mask so I just duplicated the shape layer, deleted the ellipses, and then used expressions to tie the size of the bottom rectangle to the size of the top one and use the bottom one as the mask. Maybe there's a better way to do that.
The only catch is I can't figure out how to drag a side of the rectangle with my mouse instead of animating the properties. I realize this might be a dumb question but is there any way to be able to grab a side and drag it out instead of sliding the size property? Right now it's animating the scale property which skews the ellipses instead of the size property.
As with all things shape layers - if something doesn't work as expected, add another level of grouping to get an extra transform. Parametric items don't have transform handles, anyway.
You have to change the size of a rectangle using the size parameter. If you try and click and drag you change the scale.
Yeah, do you mean pre-composing to get another level of grouping? Or parenting the shape layer to a null? I'm still learning my way around AE.
RE: Rick Gerards addition.
There are two levels of 'size' within a 'Rectangle' shape.
Opening up the path you will see you can alter the values for the rectangle.
And within Transform: Rectangle you have the 'scale' of the rectangle (and all paths in that 'level of grouping'.
Dragging vertices will change the 'scale' value only and alter all paths in that level of grouping.
Tip especially if you are getting in to expressions
Sometimes its easiest to set the rectangle to 100,100 - then your 'scale' will be an identical value to the rectangle 100,100 if required.