Uninstallying older versions leaves things like the Plug-ins folders AFAIK. Are you sure there is nothing you need to keep CS5 for though? Twain drivers, other Plugins etc?
All Adobe upgrades are FULL installers. They do not rely on a previous installed version, nor do they touch said previously installed version. They only read the serial number from the prior version and if they find none they will just prompt you for the serial number.
DO NOT drag things into the trash. Use the Adobe UNINSTALLER.
The potential problem is that the uninstaller may mess with the newer version, as they used similarly named support files.
It's always better to uninstall the older version first, before installing the upgrade.
At least on the Mac, you should NOT copy or move into CS6 plug-ins from prior versions. That's a big no-no that causes serious problems. Plug-ins should be installed FRESH from their installers into CS6.
Some users develop file association problems when uninstalling a previous version. File association, believe it or not reverts to the uninstalled version.
If you really do not want the previous version on your computer the safest route is to uninstall both, run Adobe script cleaner, then reinstall CS6.