You're trying to do everything on one layer. Roto, Precompose, then time remap or some combination of the same.
Yeah i kinda thought it would be something like that.
But how do i do it in seperate layers?
i'm having this same issue. did you figure this out bro?
you need to pre-compose first, do your time re-mapping inside that pre-comp, then apply the roto on your pre-comp.
Also, for the freeze, make a keyframe for what you want to freeze, copy-paste it to where you want to resume, and delete the last keyframe of the time remap (the one that AE puts automatically at the end of your footage when you apply time remapping). The way you described it still introduces a (very) slight motion which might give you unpredictable results - you spread a 2-frame motion over a few seconds. Repeat the process for other layers (no need to pre-compose if you don't roto) but make sure you keep them in sync.