And I think I found the issue since the layer has a mask on it, I get that result. If I remove the mask it doesn't shift right.
Turn off the fast preview at the bottom of the comp. Sure, some weird OpenGL, but other than the usual "update your graphics driver" I doubt you can do much about it and only revert to software rendering mode for the tiem being.
Hrn when working in the Layer Windows I don't see a place to turn off the OpenGL stuff like I do in the comp window. I can solve the issue by setting MASK to NONE and then the shift doesn't occur. It seems to be related to rotobrushing a clip when a mask is applied.