What would be the proper way to set up a trap for two mid-tone colors in IL (CS5)? As an example, if one color was cyan and the other color yellow, the resulting trap would be green, which would not be desireable I would think. Any other way to do it?
It's a value judgement.
What are the tint values of the cyan and yellow?
Based on that, which would be the most distracting:
The general rule is to spread the lighter ink into the darker.
To see what Illusrator's default settings would recommend:
1. Set View>Overprint Preview to on.
2. Draw two overlapping circles. Fill one with your cyan value; the other with your yellow value.
3. Group the two paths.
4. Effects>Pathfinder>Trap. In the dialog, set the Thickness to something ridiculously huge, just to make it easy to see. Turn on Preview.
Assuming your original percentages are 100% each, you'll see that AI's built-in Trap effect would spread the Y into the C, but with a reduction in percentage(s). The principle is to reduce combined percentage so as to approximate the overall tonal value (lightness/darkness) of the darker color, making it less obvoius.
5. Object>Expand Appearance. Ungroup a couple of times.
6. Select the trapping object and check its values. You'll see that spreading Y is not 100%, but 40% (middle screenshot).
Now, I don't know about you, but to my eye, also reducing the C to, say, 70% is much less obvious than the results of the Pathfinder Trap effect (bottom screenshot).
As with autotracing and blend settings and color calibration (so-called "color management" as opposed to color correction) and other things, a little bit of human discernment (laced with a healthy measure of scepticism) goes a long way. Blind reliance on auto features often yields inferior results to a thoughtful, but common-sense manual override. So pay attention to the various settings in the Pathfinder>Trap dialog and experiment with them. This can help you understand the principles at play. Then don't be afraid to use your own judgement.