If AE is up to date and your composition preview is set up properly then you should have no problem with the default values for inner and outer glow. Your screenshots would be much more useful if you included the entire AE UI and had selected the layers you are having problems with and pressed the U key twice to reveal all modified properties. There should be no need to pre-compose or nest your text layer. Even if you wanted to do that you should have no real problems if you apply the default values for the layer styles. If the values are at the default settings pressing the U key twice will not reveal the layer styles.
The first thing I would try to do is reset any modified properties. I cannot duplicate your problem unless I set the Technique in the layer styles to Precise or jack up the Spread to ridiculous amounts. I think it's just user error..
I have been running into this problem for some time now. Whenever I combine layer styles and blurs, the layer styles tend to get overblown and I don't get the look I want.
Layer styles are always applied as the last step after all other effects and operate on specific blending thresholds. Even a layer with 0.001 percent opacity will render some of them opaque. There is nothing wrong with how this works, you jsut need to understand the rendering order and learn how to create these things using standard effects and matte techniques or as a minor, animate the effects parameters so they only kick in when you actually want them. I rarely ever use layer styles in AE for exactly those reasons - they are more trouble than what they are worth and only mess with your other stuff and in the end look completely iffy like in your example. In the end, an inner glow is nothing more than an inverted and blurred layer matted out with your text and that's easy enough to do...
Whenever I combine layer styles and blurs, the layer styles tend to get overblown and I don't get the look I want.
Mylenium explained it - layer styles are always the last to render in the render order chain. you can even see it for yourself if you look at the content structure of any layer from top to bottom.
this means any effect you apply, or even motion blur as a result of transform movement, will be rendered before. this usually causes all kinds of nasty artifacts. BUT you could use that to your advantage at times, and I do use layer styles from time to time fully aware what to watch out for. you could of course, as M suggested, dismiss the layer style and create the look you want using effects and other procedures.
I tried a new approach today and precomposed my text and put my blur inside first, and then added the layer style on the outside, but the issue still happened.
you are very close to solving it. what you should do is precompose the layer with the layer style inside and the effects on the outside and everything will be normal. you see, pre-composing is also a way to change the render order.
so if before the render chain was:
2. Layer style
1. Layer style
I just want to clarify: I want to use Layer Styles in AE this time, because I want to use the Animation Preset/Text/Blur/Evaporate Effect. I would otherwise avoid this headache by just making the text in PS. ... I feel like this shouldn't be an issue to begin with, though.
I tried both Myleniumand Roei Tzore'ssuggestions. I can't get the Evaporate effect after I've precomposed the layer, as the top layer is now no longer text.
I'm including additional screenshots. In these images, the text is still on the "inside":
Image 1: Layer Styles with full UI (edit: this should be artifacting... I forgot to turn the effect on when I took the screenshot).
Image 2: My Evaporate (Preset/Blur) effect (starting point)
Image 3: My Evaporate effect (end point)
My After Effects software is completely up-to-date, by the way (listed my specs in the original post).
I can't get the Evaporate effect after I've precomposed the layer, as the top layer is now no longer text.
Ah Ha. you are using the text animators (as you have mentioned and I did not notice - my bad). I was talking about effects. the text animator blur, like effects or any other procedure in the layer chain, will also render before the layer style because it belongs in the text content.
but unlike effects, you can't create an instance of a text animator on anything but the text layer. so you can't use a pre-comp to change the render order. in addition, effects will always render after the text blur so even with effects you could be in the same situation where you will have artifacts as you are blurring something and then you add an edge to a blur...
Your options are:
1. keep the layer styles, lose the evaporate effects, and pre-compose (move all attributes) and use effects for blurring the text in a different manner (blur a compositing mask on the layer to control certain amounts and range for example)
2. lose the layer styles, keep the evaporate effects, and use effects that could be more forgiving in artifacts or matte procedures to create outer and inner glow.
3. change the appearance of the layer styles so they won't have a harsh edge - I believe "spread" in the Outer Glow and "Range" in the Inner Glow are causing most of the problems.
as for Option 2, maybe just for the challenge, Here's an example of recreating the inner and outer glows with effects and not layer styles all in the same layer. there is no "inner glow/shadow" effect in Ae, but you can through a series of effects create the same look. I have typed UU so you could see all the modified properties for the effects:
as I said - these effects, like the layer styles, also render after the blur - but I found them more forgiving in artifacts.
Thank you Roei Tzoref! I ended up going with the 3rd option. Changing the range and spread was all I needed to do. It's not as stark as I really wanted, but I'm able to get the look I generally want, still. Thank you for all your help and advice!