Is there an easy way to convert those to CMYK without opening each one in PS?
Yes, on export or print.
The numbers you see in Separation Preview are conversions to CMYK via your document's CMYK space. When you export to PDF choose Convert to Destination from the Output tab with Document CMYK as the Destination profile and any objects or images that are not in your doc's CMYK space will get converted to document CMYK on export.
You can't do it with seps preview, but you can set a preflight condition to flag them. You can also include color space in the things you show in the links panel, but that won't help with pasted or native content, so the Preflight option is probably better. Each flagged error in preflight will be acompanied by a link to the problem.
You can do an RGB to CMYK conversion during export to PDF, but you can't convert the images otherwise directly in ID.
You can one of several methods to find out:
(1) You can use the Links panel. Open the Links panel. Go the panel menu and choose Panel Options. In the Panel Options dialog, Under Show Column check "Color Space". Now when you view your Links panel it will show the color space of all linked images.
(2) You could use Preflight, and set up a Preflight Profile. In the profile, go to the Color section > Color Spaces and Modes Not Allowed, and check RGB. Then any RGB images would be flagged.
(3) If you need to know BEFORE you place the images, use Adobe Bridge. Open the folder where the image are. Use the Metadata panel and view the color space of the images.
Thanks Steve. (2) worked perfect.