Quote from the CS4 reference:
For Flatness, leave the flatness value blank to print the image using the printer’s default value. If you experience printing errors, enter a flatness value to determine how the PostScript interpreter approximates the curve. The lower the flatness value, the greater the number of straight lines used to draw the curve and the more accurate the curve. Values can range from 0.2 to 100. In general, a flatness setting from 8 to 10 is recommended for high-resolution printing (1200 dpi to 2400 dpi), and a setting from 1 to 3 for low-resolution printing (300 dpi to 600 dpi).
As far as I know:
1. The parameter flatness is used in Photoshop only for clipping paths.
2. Clipping paths are used by output devices like printers.
3. Clipping paths are in programs like Photoshop or InDesign smooth Bézier curves.
4. dpi (dots per inch) refers to output device dot resolution (not ppi!), for instance
1200dpi for an inkjet or toner printer and 2400dpi for an imagesetter for offset
5. Rendering the clipping path, the smooth Bézier path is replaced by a sequence
of straight line segments.
6. The max. deviation from the smooth Bézier curve is defined by flatness.
For 2400 dpi we have approximately 0.01mm dot distance. Flatness=10 means,
that the accuracy is not worse than 0.1mm. Fairly sufficient for a clipping path,
but more important is: the path appears smooth despite its rendering by a polyline.
7. No value for flatness will result in printer default settings, which are derived
from the actual resolution in dpi.
8. flatness is used for any rendering of vector paths by PostScript devices, for
instance for vector graphics by InDesign or Illustrator.
9. All this is not applied in Web images. The mentioned "300dpi images" may be
(correctly named) "300ppi images", but that's irrelevant. Only pixel numbers
count, because one source pixel is represented by one monitor pixel.
(Let's hope that the images are not scaled by the browser using width and height
with new numbers, which results in ugly resampling by nearest neighbour)
Best regards --Gernot Hoffmann