Keeping compatibility for both SDK 3.5 and SDK 4 in the same code-base can be quite challenging given the fact that Flex 4 has syntactic changes in MXML and CSS.
My suggestion would be to separate them and treat them differently. Keep pieces that do not rely on framework specific features in a common library.
P.S: As Gurdeep suggested, you'll get better answers for this on the flex general discussions forum.
Thank you for your suggestions. But isn't conditional compilation a flash builder feature, rather than a flex feature?
The actual flex compiler resides in the Flex SDK. Flash Builder makes API calls into the SDK to compile.