I just installed Lightroom 5 on a second machine from a trial version. It asked about updating to a newer catalog. I did and it said "the Lightroom Catalog cannot be opened because it is too new to be used by this version. Would you like to select a different catalog?" Why is this the case?
If you created a catalog with the Trial Version (most likely 6.8 the 'latest') then this catalog is not backwards compatible with Lr5 versions.
If you upgraded an existing Lr5 catalog to work with the Trial version, then you should have two catalogs- the Lr5 catalog, and a Lr6 catalog that was upgraded from a copy. I think this is your situation. If you do not upgrade to v6 then you will have to find and 'open' the catalog that worked with Lr5.
Note: the catalog names may not indicate the compatible version number, but upgraded catalogs are renamed with a "-2" to indicate the upgrade.
So for example: a Lr5 catalog = My Lr5 Catalog.LRCAT
An upgraded Lr6 catalog = My Lr5 Catalog-2.LRCAT