Create a layer that just has the content that you want CAF to use to fill your selection.
Confused by your intention more than what you are asking...
What is the desired outcome? Have you tried using the background eraser tool in combination with a selected area (like when using the lasso)?
You can adjust the size of the eraser and also use tolerance settings and the brilliant "protect foreground colour" to improve your erasing ability.
Do you even need to erase [EDIT] destructively? Can you show an example? Could you not just non destructively accomplish your task or does it need to be a permanent alpha transparent photograph.