Selecting fine details whithout "auto-select"

I'm not sure exactly if I understand what you're trying to do, but if you hold ALT down there will be a deselect option that will enable you to deselect the parts of the photo you don't want selected. There is also the Lasso Tool that won't automatically select anything you don't want selected.

I would suggest using the Lasso Tool instead so you can select and follow the details you want under your own control. That way you can select the area on your own as opposed to Quick Select automatically selecting the are for you. You could also try checking Auto-Enhance when using the Quick Selection tool and it will select with more precision.

The Magnetic Lasso Tool works great as well.

The Quick Selection tool's precision is greatly enhanced by use of the Refine Edge command in its Options bar -- particularly the Edge Detection feature. I suggest you give Quick Selection another try (along with Refine Edge) before ruling it out. A sample of the image for us to see may prompt other suggestions.