Unless you are math genius who can create his own interpolation algorithms that will never work. Even if you were to do that you'd need to find a way to actually get the scene to respond to such changes, which means you would use projection mapping on geometry or point clouds and inverse-stabilize it. That's simply way outside AE's domain and would require Nuke or at least some work in a 3D program. At best you could possibly stabilize segments in your long shot individually and then use warping and morphing tools like RevisionFX' RE:Flex to stitch them back together, but even that will be a lot of work. Sorry, but it's probably safe to say that this is one of those cases where you shot yourself in the foot by not planning the shoot accordingly and using a steadycam.
The 3D camera tracker is not for stabilization and cannot be used for that at all.
If you are having warp issues with your shot then you need to correct the rolling shutter first, render a digital intermediate, then try and stabilize the shot. Rolling shutter problems with lots of warping in a 4K shot tell me the shot was not taken with a professional 4K camera.
You must learn this film fact - some shots will not successfully stabilize, most will not stabilize as much as we would like, and a few will stabilize magnificently. If you want a stabilized 6 minute shot you need to start with pre-production planning and then be lucky enough to get great, make that extremely great, camera work by an experienced operator. I would not even attempt that kind of a shot without a steadycam. Even with an experienced operator this shot would require rehearsal and planning to set up properly.
Trying to stabilize 6 minutes of hand held footage with a walking camera making turns every few seconds is nuts. The only way you are going to have any success is the cut up the shot, fix each part, and then put it back together again.
For example if you have 15 seconds of the shot walking in a relative straight line, then 3 seconds of turning a corner that is rough and then 6 seconds of relatively good footage you need to cut the footage into 3 pieces. Stabilize the first section as good as you can selecting stabilize only. Do the same with the second and third part, then match up the last frame of the first part with the last frame of the second part and the last frame of the second part with the first frame of the last part. The shot is going to slip in and out of frame because of the stabilization so to fix that attach all of the parts to a null and scale the null and add a few keyframes to the null to smooth things out and fill the frame. Some of the really rough parts will probably have to stay really rough. If you cannot remove the rolling shutter warping then you are probably sunk.
Note: your digital intermediate must be a frame based production codec. NO mp4's allowed there.
Thanks Rick and Mylenium, I really appreciate your comments.
It is actually a motion control shot and then in the middle it transitions seamlessly to I think it's a Glidecam (or another alternative to steadycam) shot on Epic.
The motion control section is great, but after the handover things get rough.....
I am now trying to work on the shot in sections like you suggest and hope this will improve some of the "human operatorness" which is inconsistent with the first part of the shot.