Some footage just does not slow down well. Here's what I would do. First, reset the time warp effect to normalize all the settings. Second, set the speed. Third, step through the footage one frame at a time until you find the frame with the most problems. It is here that you should start experimenting with things like block size, smoothness, filtering. When you get your best looking frame it is time to do a ram preview.
Some elements in the shot, like a hand moving for example, may require some roto or other techniques to create a matte layer. With some footage you will get a better result by simply using TimeRemapping with frame blending turned on. Sometimes Twixtor is the only solution. Twixtor has a lot more options than TimeWarp, but neither of them offer a one click solution to difficult shots with lots of edge detail.