Please try to deselect "Create PDF Compatible File" when saving Illustrator file.
If you select this check-box (default is selected), Illustrator saves PDF data with embedded EPS data.
maybe your linked EPS data is compressed, and then 20MB on the desktop, but not compressed in PDF or less compress cause made huge file size.
As mentioned already, with certain options data will always be embedded. That would especially be true if the linked file is part of a larrger construct with belnding options, clipping masks etc., requiring to produce additional clipping paths and objects when creating PDF data.