There's a dirty little secret with 1-pixel strokes: They're not actually center-aligned. (How could they be and still stay sharp?) So your single-pixel stroke is actually aligned to the inside. And it's anti-aliased. What you're most likely noticing is the lightly-colored fill showing through the semi-transparent portions of the stroke.
A couple ideas on how to fix this: a) Set stroke width to 2px, center-aligned, and choose "Fill over stroke" under Stroke Options, b) Leave stroke width at 1px, align outside, and decrease the object dimensions by 1px, c) Set stroke to None and add the stroke using Photoshop Live Effects. Somehow, no matter which position is chosen for stroke alignment within Photoshop Live Effects, fill bleed-through along the outer edges is not a problem. However, every solution will change how the circle is rendered somewhat, so finding the best solution requires trial and error.