Hi Elise: notice that you are in the InDesign forum and not Photoshop. That said yes, you can change images from RBG to CMYK in Photoshop as you mentioned. The fastest way is to write an Action, and run it on all the files in a folder, or all open files, or all files selected in Bridge. Alternatively, if you are printing from InDesign and need separations, and if your printer supports it (lots of "ifs") you can output to Composite CMYK while printing. I think you are better off doing it in Photoshop.
There are not many reasons to make conversions to CMYK in Photoshop. Assuming the destination CMYK profile is the same, the color managed conversions inside of Photoshop, on export from InDesign, or printing from InDesign or Acrobat wil produce the same output values.
There are downsides in making the conversion early in the workflow, when you might not know what the correct CMYK profile should be and once the conversion is made, the conversion to another CMYK can produce inferior results.
Leave images always in RGB with Color Profile. If you export for print with PDF/X-4 and X-3 they can remain in RGB, if the printer insists on CMYK you have to export a PDF/X-1a which converts everything to CMYK.
Using RGB images with transparency increases your flexibility and even the output quality.
In Photoshop, when you work for print, you can turn on the proof preview for your CMYK color space, you can even see any plates. Even if the Photoshop file includes a spot color, the image can remain in RGB.
But when you use technical drawings like a logo or in most cases an AI file from Illustrator, you should work for print in CMYK.