I've been working with the rotobrush in After Effects for about a year now and never experienced this problem. I am using After Effects CC 126.96.36.199. When I add the composition to the render queue and begin rendering, it is starts to show areas that I did not select and getting rid of areas that were selected. The export setting are Quicktime file (.mov), RGB+ Alpha, Animation codec.
Basically, the final file doesn't look like what I originally selected with the rotobrush. Any idea what is happening? I already tried restarting the program, tried using .mts files, tried using .mov files. Again, never experienced this issue before in the year I've been using this feature.
Two possibilities. Cash and mismatched frame rate. Check those out.
In terms of frame rate, it gives me a warning on a regular basis that the frame rate is mismatched. At first it told me to change it to 59.97 and then told me to change it to 19.98. The footage was shot in 29.97 and that is what it was set to. Also, 19.98 wasn't even an option.