Duplicate the layer, remove the keying effects, and animate a garbage matte around the tie.
Keying is as much art as science. The best keying rarely results by applying one copy of one keyer to the original footage without additional help in the form of masks , as Dave says, or more complicated and layered effects. You can take a copy of your footae and dramaticall boost the chroma or the green channel, use that to drive the keyer of your choice, and extract the alpha to cut your hole in another layer.
There are many great tutorials for sophisticated keying with less than superb footage. Just takes some digging.
Message was edited by: bogiesan
You can use masks in Keylight (and many other keying plug-ins) to retain the foreground. That's the direction I would start with. However, if we could see a screenshot of your project we could give you more specific suggestions.