I'm new to After Effects, so it's difficult for me to tell what the relevant information is here. I'm also not sure what all is plugin-specific, so please bear with me.
I'm using the "Saber" plugin from video copilot on a mask that was autotraced from a PNG. It is correctly applying the "saber" shape around the mask, however, when I try to use the "Alpha mode" setting, which lets the mask actually mask the generated effects, it doesn't do anything. I'm not sure if having these layers in 3D is part of the problem, since I've gotten this working fine when using 2D layers only (and a normal circle mask).