Turns out the problem is that if the application is crashing out, then the response never comes back to the player, and it doesn't fail nicely. I guess this is somewhere between a feature request/bug report then.
What's the best way to follow up on that?
Are you working with the 4.0 release or one of the newer release candidates for 4.1? I ask because in 4.1, we can now use the global error handler (if you are working with Flash Player 10.1 or newer) and have a fix for this.
I'm on 4.1RC2, but I've worked my way all the way up and down from 4.0 through 4.1RC2 and saw the same thing.
What do I need to turn on to see the global error handler working in the 4.1RC2 stack?
In rc2, you can call a method of the core:
core = new FlexUnitCore();
core.addUncaughtErrorListener( systemManager.loaderInfo );
core.addListener( CIListener );
core.run( SimpleTest );
You need to pass it the loaderInfo object... (that is how Flash Player catches top level errors)
So, the code above will work in a Flex app.