If you're making the PDF in InDesign, choose a PDF preset like Smallest File Size.
...either you change the PDF presets when exporting (as Steve already said),
or you save the pdf as 'pdf, optimized' (using a different filename ) and change the settings there.
Try out different settings for pictures' solution and quality, which will have the biggest effect on file size (and on picture quality, too...)
Just using 'Smallest File Size' could make your pics look rather bad.
Another solution would be to leave the PDF larger (so the pictures look good), but to store your PDF on a service like Dropbox, and send a link to your future employer. Of course, this assumes they're willing to accept it that way.
..right, if the pictures/scans in the portfolio should not get too much compressed, that's a good option.
But you do better ask them first; if they get 10 other portfolios via mail, they will not necessarily go to watch the eleventh one on Dropbox..