what am i doing wrong?
Arguably the deletion is not a good idea when a Layer Mask might also work.
Select > Deselect (cmd-D)
might be what you are looking for.
But I think what you really should be doing for this is using the Spot Heal brush which is similar to the Clone Stamp tool (in terms of results not usage).