If I have a Premiere 5.5 project with some clips being replaced by After Effect comps. do I have to have the After Effects program running when I export from the Premiere timeline?
That's what Dynamic Linking is all about.
Does it sort run a 'Headless' version of After Effects? when rendering the export
Look at your Activity Monitor. You might see an app running called "AErendercore" and/or "dynamiclinkmanager."