Nobody can tell you anything without proper info about your project. Provide system and version info as well as comp settings, layer switches used and so on. Screenshots tell a lot.
I'm on Win 8.1. AE is latest version.
32 gigs ram
There's a background that I would like to be able to show up. But when the mask is set to subtract, the alpha transparency goes away. The other mask settings like add don't affect it. I've tried pushing lots of buttons to no avail, so here I am. Screenshot included.
When you set a mask to subtract the area outside the mask becomes opaque and the area inside the mask becomes transparent. Without the mask showing in the comp panel I can't figure out what you are trying to do.
Inside that WTUPC comp are just two PSD sequences, (one a main render, the other a post effects pass, both carrying alpha channels with transparency) that had an alpha channel working fine without the mask. So if I add a mask, it undoes the alpha channel? So masks and alpha channels are not compatible? They cancel each other out? I can't imagine that this is the case.
Still this doesn't work. Still. Does anyone have any clue about this? Still looking for an answer? I can't even figure out a workaround.
Ok, figure out the solution. You have to go into the comp and mask out what you want, then copy and paste the mask for each layer. Masking on the COMP for some psychotic reason undoes the alpha channel. But if you mask INSIDE the comp, the alpha channel is retained for the comp.