As far as I know, there has never been a way to change catalog names from within Lightroom or when upgrading catalogs.
The easiest solution is to leave the catalog names unchanged, don't let it bother you.
Yes, if you must, go ahead and change the catalog name and preview folder name using your operating system (with Lightroom closed).
As DJ implied you need to change both catalog and preview names. They should be the same and do not change the extension. If the names do not match Lightroom will not see the previews and will generate new ones. Also once so changed the catalog will no longer be the default. Double click the catalog to launch LR with it or in LR choose File>Open
Once you chose a catalog to open, in the middle of the dialog box that came up next, was the path and name of the converted catalog. There used to be a button to the right of that that said 'Change'. That no longer is there with conversion of catalogs to LR6. This isn't really inside of LR, it's during the process to convert a catalog to the newest version.
Here is the current help link showing an upgrade to LR 5:
I'm not at my LR computer now so I can't inspect this dialog in LR 6, but that button in LR 5 meant to change the referenced catalog and not to rename it. It may be that once clicked the operating system allowed for file renaming in the resulting dialog box but that's not the primary use of the Change... button.
I followed the catalog renaming instructions contained in that FAQ, and it worked fine for LR6. You're not changing any folder names, just those 2-3 file names.