The gaussian blur setting is pretty high (2 cm) for your object of 3 cm. I got the same behavior. Looks like .9 gaussian is the highest it will go without the edge.
Looks like you're trying to achieve a very soft but largely diffused glow behind your object? Maybe try a separate object behind with a radial gradient on screen? I think it would achieve the same effect.
I’d say it’s happening because Illustrator is a buggy piece of crap that hasn’t been out of beta in over a decade.
I find Illustrator's raster effects terribly anaemic. I prefer the versatility of a hand-made effect. For Outer Glow I add a stroke using the Appearance panel and drag it to the bottom of the stack of appearances. With the stroke selected I then apply a Gaussian Blur effect. This is more versatile than Outer Glow because I can alter the size of the effect using the stroke weight without touching the blur. I can also use a brush for the stroke or apply other effects, like Scribble.
Bumping this as I've just had the same issue in Illustrator CS5 and this is the only place on the net where I've been able to find info on this issue.
The solution for me was to drop the Document Raster Effects Setting reolution bit by bit until the problem stopped occurring. In my document the magic number was 262ppi.
To me this is a bug as there is no time when you would want this effect to occur when adding a glow to an item (text in my case). Present in Illustrator CS5 (15.0.02).
This occurs to me couple years ago. I remember there's a formula where you can calculate the amount of blur/glow pixel in relate to DRES you need to set in order to get rid of the square edge and retain your effects appearance. The problem is I don’t remember what was the formula and yesterday I ran into the same problem again!
Better yet, here are some sample files. These file usually come from design agency / artwork agency.
I remember this issue has been brought up a couple of time in the prerelease but never gotten a fix.
Just a little reminder: This behaviour IS a bug. Moreover, it's a serious bug because the unwanted stuff will print, it's not just a screen issue.
As far as I remember, it has been introduced in AI CS2 or possibly AI CS3. Specifically the Outer Glow problem.
You don't get the issue in previous versions of Illustrator.