It would be hard to give someone the go ahead to just delete a bunch of files, withot being able to see and discuss every one, but I can say that as far as settings and such go, the versions of Photoshop keep themselves completely separated from one another.
If ever you're unsure, rename the folder(s) and see how things work.
You can delete the Bridge cache. If it is needed it will be rebuilt.
If you have installed CS6, you might want to use edit>presets>migrate presets to import all your presets that have been saved in their default (roaming) folder, then go ahead and delete the ones listed with the old version number IF you are sure you have all your saved actions, brushes, and other presets.
Thanks everyone for all the advice. I've moved the cache to another drive, copied all my presets and deleted the old CS5-related folders. Everything is working OK and my C: drive is no longer being filled up with cache files.