That's the difference between a simple Flash based SWF file
and one based on Flex. The latter would load, first and foremost,
the minimum of the Flex Framework libraries needed to run a Flex
app. The minimum size of the a Flex Framework that you absolutely
can't do anything about is around 135kb.
So essentially your 'empty project' consists of Flex
Framework itself. Try adding few components and you'll notice that
it won't grow exponentially.
You can use RSL (Runtime Shared Library) to potentially
reduce the size of the Flex swf, at the cost that the first time a
user sees ANY (not just your) Flex App with RSL enabled, they would
need to download 500K.