Go Modules > Photo Gallery
Select your specific gallery
Then Manage Items
There you will see the items in your gallery. Select the item and then underneath in the Description Field type in your caption. Hit return to save the description. Select another and follow the same process.
The photogallery module does not allow for such captions to be under the thumbnails in list view. The description field is just shown when the image is clicked.
You can probably use some jQuery to add a div under each image, then get the alt text from the img (which is the description) and place that into your new div.
Hope this helps,
Chad Smith | http://bcgurus.com/Business-Catalyst-Templates for only $7