I select and mask the rider. Next I blur the background using Gaussian Blur filter.
Have you removed the foreground element from the background you blurred?
Could you post an exemplary image (or lores version) and the result you got so far?
Sorry, what with the long title I had neglected the body of the message and your statement
The only thing that works is to go around the edge with the Clone Tool. This is time consuming and not totally successful.
seems to answer my first question.
Did you do that before or after blurring?
Have you alternatively tried using Content Aware Fill?
Create the mask as you did, but make a channel out of it. Use Lens Blur and use the mask/channel as the source. This will keep the edges from bleeding over. You can't add as much blur as you can with gaussian blur, but you can apply the filter several times.