Can we see a raw shot without the keying work?
What format/codec is the footage you're working on?
Can we see the shot in motion?
Sorry forgot to mention that you will need to full screen the motion footage to notice the noise/chatter.
Cleaning up chatter takes time and testing. You need the cleanest matte you can generate. If the footage is highly compressed or has limited color sampling (4:2:2 or 4:2:0 as opposed to 4:4:4) then it is going to be more difficult. Unfortunately, without actually having a copy of the footage in it's original format for testing we can't give you the exact settings. Try experimenting with the advanced matte cleaning features. Check the Combined matte often. You may even have to add additional matte clean up effects to the footage. Keying, really good keying, has never been an easy 1 or 2 click proposition.