It's hard to know what you are doing since we can't see it. Perhaps some screen shots would help.
Are you setting the layer's blending mode instead of the mask mode? Is it a cache issue? We don't have enough info.
A mask mode is usually add by default, so I don't know why you're having to change it.
What else is in you scene?
What exact version number of After Effects are you using (down to the decimal points)?
Masking like you describe is a pretty basic After Effects function. If a step like this is causing you issues, you should really step back and start here first: Getting started with After Effects. Trust me when I tell you that it will help you avoid much frustration.
Also, try searching for RELECTIONS in various AE tutorial sites. There are even several plugins you can buy that do this for you. While figuring it out is a good exercise, if you're doing several comps that require accurate depiction of reflectivity and interactivity with light sources while a camera is moving through a scene, a plugin will save you DAYS of frustration.