If I understand you correctly, you want the mask to scale down without affecting the layer overall?
If so, you can set keyframes for the mask path property to move it as well as to scale it. In order to scale the mask down proportionally, try double clicking one of the mask vertices and using the "free transform" box that appears. That should give you what I think you're looking for.
thanks ! that hit the nail on the head