I know EXACTLY what you mean. I was doing something exactly on those lines just yesterday.
It's not trivial, but it's doable.
For hard "spots", you can try the Spot Healing Brush first.
But here's a neat trick I sometimes use that can nicely increase the overall smoothness:
1. Select just the skin area.
2. Contract the selection a bit and feather the edge, just to make sure you don't mess up the edges.
3. Filter - Noise - Median, and adjust the radius to be fairly large. Watch the image to see the smoothness level.
4. Edit - Fade and knock back the effect, so that it's not perfect, but makes a subtle improvement.
Then, for gentle darkening lightening, try the Burn and Dodge Tools, set to a low Exposure level (and possibly with Airbrush Mode on). Work gently.
Cloning with low opacity can sometime help as well.
I have some photographs where the skin of the model's leg is not of uniform brighteness. Without blurring it, I have tried the clone stamp but it is very difficult to not introduce further variations in brightness.
What is the best way to make skin uniform in brightness (actually, the same color!)?
onOne software produces a range of plug-ins for Photoshop, Lightroom, and Aperture. One of the filter effects, "Model Skin," would be ideal for what you want to do, and was included in the FREE sample version of Perfect Effects v2.6. Perfect Effects v3 Free Edition is now available; I'm not sure if "Model Skin" is still included, but it's worth a download.
As there is hardly a task that can not also be done with Smart Objects I would recommend …
• Lifting the skin onto a Layer of its own (with a fairly good Selection naturally)
• converting to a Smart Object
• applying a Layer Mask of the current transparency
• applying a big-ish Gaussian Blur to the SO (not the Mask) and either using a noise-Pattern Overlay set to Linear Light and reduced Opacity or the Filter Add Noise
• reducing the Layer’s opacity
Edit: For additional masking a Layer Mask on a Group can be used.
The result will likely not be better than with Noel’s method, but at least the file will be larger …
Sometimes just painting on an additional Layer (with the appropriate Pattern Overlay to fit in with the image’s noise/grain) can help, too.