Up until LR5 the "UserComments" EXIF field was imported into the Caption field of the metadata.
Hmm, in my LR 4.0, the EXIF:UserComments field is not imported into the LR Caption field. Are you sure that's happening with your LR 4? Which version and platform?
LR 4 and 5 do read the EXIF:UserComments field, but they display it in the Metadata panel as "EXIF: User Comment". Below is a screen shot of an example showing the field being imported and displayed.
I'd be surprised if LR 4 or 5 ever imported its Caption from EXIF:UserComments. Adobe supports (imperfectly) the industry standard Guidelines for Handling Image Metadata (Metadata Working Group), and it specifies that the caption should be stored in IPTC:Caption-Abstract, XMP:Description, and EXIF:ImageDescription. My tests confirm that's what LR 4 and 5 do. (LR reads the caption from those fields in the order above, giving preference to the first non-blank field it finds.)
When I bought my Canon 30D in '06 I started putting ID info in the comments field using ZoomBrowserEX that Canon supplies. As my picture volume increased I found 2 problems with my workflow: 1) putting the same or similar ID into a large number of images was a real pain, and 2) If I needed to edit my comment ZoomBrowserEX tended to put the edits into a XMP sidecar file for the directory rather than individual files so moving an image left the ID behind in the old directory. To more automate workflow I started using exifutils from www.hugsan.com to insert the comments into the EXIF:UserComment field using a batch file. About the same time I started using Lightroom (I believe version II) which picked up the EXIF:UserComment field and placed it in the Caption as shown in this screenshot:
I was using exifutils rather than EXIFtools which does not directly support writing multiline comments. Starting with LR3, comments inserted with ZoomBrowserEX failed to import, but those inserted with exifutils would import. As near as we could figure the end of line codes used by ZoomBrowserEX caused LR problems while the CRLF inserted with exifutils was OK. Due to several problems with exifutils I was finally able to devise a workaround for inserting multiline comments using EXIFtools with the command line: perl C:\EXIFTOOL\exiftool.pl "-UserComment<=Cmt" 1401C_0989.cr2 after creating a multiline ID file named cmt. Using EXIFtools I have been working to extract the comments inserted with ZoomBrowserEX and reinsert them using EXIFtools which is time consuming for thousands of images. LR3 and LR4 have consistently imported these comments to the Caption field as shown above. Somewhere in the last updates to LR4 or the new LR5 these comments are no longer imported. I have verified by having LR export the metadata to an XMP file, as it does automatically with RAW images, that the EXIF:UserComments I inserted is in the XMP export for both images where the comment formerly imported to the Caption and those where it does not import now. The only difference in the image structure is what formerly imported no longer does.
I think but can't guarantee I have the above chronology exactly correct. After dealing with thousands of my dad's unidentified pictures I am trying to make mine have the location and subject data physically in the image and not in some sidecar file that can get lost. I need multiline comments that are visible in Canon's ZoomBrowserEX and Digital Photo Professional as well as Lightroom. If necessary I can insert the comment in multiple places within the image, but I really need to know where in the mysterious depths of Canon's RAW files LR can pickup a comment and place it in a visible field. The Caption is sort of convenient but another may do just as well. I also need to deal with RAW files from Canon's 30D, 70D and S95 as well as whatever camera I buy in the future.
Thank you very much for your help. I am running 64bit Win7 on an I3 processor. I think I started with one of the middle versions of LR2 and have done each update as they were issued and bought the upgrades couple of months after they come out. I enjoy using Lightroom despite some of its limitations, but needing to constantly reference another program to setup my keywords is a real pain.
I no longer have LR 3 installed, so I can't verify its behavior, but I did verify that LR 4.0 and 5.5 do not import EXIF:UserComment into LR's Caption field. If you think LR 5 is doing that, it might be worthwhile to upload a sample photo to Dropbox (or similar) and post the link here, and I can take a look at what might be going on, to help you sort out your best path going forward.
It might be that somewhere in your past workflow with a variety of tools, one of the tools was also inserting text into one of the metadata fields that LR maps to its Caption field (IPTC:Caption-Abstract, XMP:Description, and EXIF:ImageDescription). Confusingly, the Guidelines for Handling Image Metadata standard observes that the phrase "user comment" has sometimes been used to describe what has historically been stored in those three fields, despite there also being an EXIF:UserComment field. Over the years, different tools have used the same term to mean different metadata fields and different terms for the same metadata field.
Going forward, I recommend that you follow the standard established in Guidelines for Handling Image Metadata. Even though the title says "Guidelines", in all other respects it's an industry standard, established by the Metadata Working Group, whose members are Adobe, Apple, Canon, Microsoft, Nokia, and Sony. LR does a reasonably (though not perfect) job of following that standard.
Under this standard, LR's Caption field is what the standard calls "Description", and it gets stored in up to three image metadata fields: IPTC:Caption-Abstract, XMP:Description, and EXIF:ImageDescription. If you're using an external tool to update the description, you don't need to write all three -- I recommend that you use Exiftool to set XMP:Description and clear IPTC:Caption-Abstract and EXIF:ImageDescription. In general (and imprecisely), the XMP metadata fields are intended to subsume the legacy IPTC and EXIF fields. The XMP fields don't have any limit on length, and you can store values in multiple languages (e.g. a description in English, French, and Swahili).
"Using EXIFtools I have been working to extract the comments inserted with ZoomBrowserEX and reinsert them using EXIFtools which is time consuming for thousands of images."
In case you're not aware, Exiftool is pretty powerful, and it will let you process an entire folder tree of images all at once. In particular, with a single command line you should be able to copy the ZoomBrowserEx comments into XMP:Description for all of your photos, skipping photos that already have a non-empty XMP:Description field.
Also, I see in your screen shot that what's showing up in the Caption field is a combination of state, city, keyword, and GPS location. The industry standards and LR support separate fields for each of those. Instead of simply copying them from a legacy field into the LR Caption field, you might consider using Exiftool to transform these annotations into the corresponding metadata fields (State, City, Keyword, GPS location), all of which LR supports well.