A sensitive movement is your mouse/pointing device, not Fireworks.
It did surprise me, but holding the shift key while moving the square adjustment point on the gradient doesn't constrain it to 15 degree increments and it doesn't "snap to grid," either. Both would be helpful. My CS3 remembers the last gradient I used so if you define a vertical gradient, that should be available next time.
Here are two things to try. You could try putting in a guideline and zooming in on each point separately. You could make an object with a horizontal gradient, then use Modify > Transform > Numeric Transform to rotate it exactly +/- 90 degrees.