I'm sure it varies depending on one's opinion but we have a policy that they should be under 1- 1.5MB. This isn't always posible of course. Our PDF's are mostly informational and contain little or no graphics.
I would say that it greatly depends on the PDF. If you get up over that size, you should do your best to optimize it for your users. Sometimes you can only do so much.