What, specifically, is the effect you are trying to achieve?
I'm guessing, but I think what you want is an oval section of one image on top of another image? If so, then the lower image doesn't need to be masked and doesn't even need to be considered. Think of that as a background.
Try this instead:
- Turn off visibility of the lower, background image, so it doesn't distract you.
- Make a white vector ellipse with the proportions you want and position it over the area of the image you want an oval of. You can set the opacity of the vector ellipse to help with positioning, but then make sure to set the opacity back to 100%.
- Select both the image and the ellipse, then go to Modify > Mask > Group as Mask.
- Turn on the visibility of your background image and position your masked image. (I think this gives you what you want.)
Your vector mask needs to be white in this case. A white area in a mask will make the masked object opaque. A black or transparent area in a mask will make the masked area transparent. A grey or semi-transparent area in a mask will make the masked object proportionally semi-transparent.
Is this close to what you want?
Thank you for your help! It's taking me some time to understand all the tech terms, but it's slowly sinking in.