Show us a screenshot of the exact settings and a reference frame befor the keying. I get that ugly feeling that your greeen just isn't the right green and that therefore you're reaching the limits of what teh effect can ahndle automatically...
What you have here, is the real worl dof colour keying!
It is very rare that you will have source footage that can be easily keyed in one or two clicks. There are many, many reasons why the footage can be so hard to key - ranging from lighting issues, to footage source type etc. What you would need to do now, is do multiple keyeing techniques and passes to get the best possible matte. There really is many different ways to go forward now, and it is going to be up to you, and your time available what you will choose.
The easiest would be (off the top of my head) to duplicate the layer, mask out different areas, (lets say his right elbow for one layer, and his left arm for the other) and key each piece seperately, with its own "Best" settings. sorry, keep a copy of the the orignal as the background layer, and key it out as well. Then by setting the layers inthe right order and feathering here and there, you should be able to build up a very good matte.
As I said, this is only one possible way, you might then want to use your built matte as a track-matte. You might like to try and use Aharon Rabinowitz' techniques to get a nice tight matte to work on - http://library.creativecow.net/articles/rabinowitz_aharon/junk_mattes.php
Search for more tutorials as well - if you have the time and money, get some books - http://www.betterworldbooks.com/green-screen-made-easy-id-EB2370003378283.aspx
Well, I hope that helps?
It would also help to see a shot with NO effect applied to the video.
Proper lighting is one of the most important aspects of getting a good key, and the phrase "I even have a couple of light sources" doesn't really describe how well the shot is lit.