Going from 5.1 to 5.7 is not an upgrade, but rather an update. It doesn't involve upgrading the catalog. Have you used the trial version of Lightroom 6 or CC? If you have then you might be trying to open the updated catalog that was created. You'll have to go to your folder that contains your it catalogs and double-click on the older version.