Good question. I hate to give vague answers, but the answer is: it depends a lot on what you're doing.
The main difference between Illustrator files and Photoshop files is that Illustrator files tend to contain mostly vectors, while Photoshop files tend to contain mostly bitmaps (although both can contain either!). Roughly speaking, the size of vector artwork is proportional to its complexity (i.e. lots of paths, gradient meshes, many path points, many objects). Roughly speaking, the size of bitmap artwork is proportional to its pixel dimensions (i.e. width and height). A giant rectangle with a simple gradient is much more efficient to store as a vector, but a 32x32px icon with lots of detail (e.g. an application's icon) is often more efficient to store as a bitmap.
So, if your design consists of large, simple, geometric patterns, you will want to use vectors (and thus Illustrator). If your design consists of little nitpicky graphics and photos, you will want to use bitmaps (and thus Photoshop). For any realistic website or application, you'll probably want to use a mix of both.
Here's the bottom line: my advice would be to use whichever tool allows you to express your design quickly, and worry about optimizing the SWF size later (i.e. when you're about to deploy it)...you can always revisit the decision about how to represent an individual graphic.
So, basically, Illustrator will still produce smaller SWF file sizes with Flash Catalyst due to the fact that it's line art instead of Photoshop because it's pixel based, unless you import bitmap images into Illustrator which would defeat that line art only rule to a degree.
This is pretty much what I expected. I completely understand the difference between the two pieces of software. I also know that most do their web designs in Photoshop now instead of Illustrator due to speed and the large amount of stylistic features of Photoshop that take too long to reproduce in Illustrator. That also means that the SWF files are going to be quite large due to the bitmap nature of the files.
This also confirms that Catalyst won't convert geometric shapes from Photoshop into true line art either, which is what I expected since Flash Professional does not do that either.
So, if we took two identical designs, basically geometrical, from each piece of software, the Photoshop version would be quite a bit larger, perhaps by as much as 10x based on the design. This is similar to what I get in Flash Professional if I don't re-do the geometrical artwork in Illustrator or Flash to save file size.