Whenever I drag the preset onto the layer (or double click on Bridge) what happens is that the text kind of flutters in.
I tried matching the outline and background colors and the effect is much better, but not what's shown in the preview. It lacks the actual flashes, precisely what the preset's about.
Though I'm new to AE I have to admit I feel rather stupid. All your answers are correct.
I gave the text a stroke with the same color as the background and checked the Motion Blur box for the layer, but didn't turn on the Motion Blur button on top. Now it seems to be working good.
Seems so simple now.