This is a long time, but I assume it's due to the file containing a lot of elements. Which version of Acrobat are you using? Which function in Acrobat are you using to make the file smaller ("Optimize" or "Reduce Size")?
I believe we were using Reduce Size. As for version, I believe its the most recent, maybe older. I work at a college, so the oldest I think it could be, would be 2013.
The execution time you are seeing is definitely not typical, and is very likely due to the inner structure of the file. For example when you flatten transparency from certain applications, you end up with thousands of little images that need to be all treated individually. Something like that will slow any process down.
If you have Adobe Acrobat Pro, you have two ways to reduce the size of a PDF file "File>Save As Other>Reduced Size PDF" and "File>Save As Other>Optimized PDF". The first one does not give you any options, with the second one you have a lot of control over what gets changed and how. The "Optimized PDF" has another feature: On the optimize dialog, there is an "Audit Space Usage" button that allows you to see what elements take up how much space, and you can then use that as input for the selections you make on the main dialog to optimize the file. This may not be any faster, but depending on what is slowing down the process, and what features you select, you may see some improvements.