I thought the term "Rotoscoping" was for the automated process, while Roto brush was what you used to define the parameters.
After reading the document I was still stuck at only ten frames so in desperation I reloaded AE. Now it's working as it should. I have no idea why it glitched the way it did.
Rotoscoping refers to the manual drawing of masks or painting on every frame. The name 'Roto Brush' is a bit misleading; it's intended to refer to something that leapfrogs over the need to do rotoscoping.