This is for the mobile app, Adobe Color. I assume you are talking about Photoshop. The Smudge tool does just that, smudge. There are some options though, up in the Options bar at the top. The options are: Mode (Normal, Darken, Lighten, Hue, Saturation, color and Luminosity. It also has a strength, Sample all layers and finger painting. My guess is it is set on Saturation instead of normal.