Apply the Image Control video effects, then click Edit Effects or right-click on the clips on your timeline and select Show Properties to access the level controls.
When I increase the brightness on one clip it appears to affect all clips. Some clips I want decrease the brightness and increase the contrast, and others I want to increase the brightness. How do I make it affect only that clip?
An Effect applied to one Clip, should only affect that Clip. Now, if you have a long Clip with different scenes in it, just use the Scissors/Razor Tool to Cut those various scenes into multiple Clips, and apply the desired Effect, and make the neccessary adjustments.
You can also keyframe the levels of the effect, just as you can any effect.