If I plug some headphones into the PC, the sound does not come out of them (from After Effects anyway) unless I restart Adobe After Effects. Is there a setting I can change somewhere (oddly enough, it says headphones already under Edit > Preferences > Audio Output Mapping).
Nobody can tell you anything without proper system info or other technical details.
Bose QuietComfort 15 headphones
After Effects CC 2015 Release
Window 8.1 Enterprise
Inetl Core i7-4650U CPU @ 1.70 GHz 2.30 GHz
8 GB RAM
NB Sound from other programs works out of the headphones, however, headphones need to be plugged in and on before After Effects is loaded, it will not output to the headphones if I plugin after I have opened After Effects for example.
Check your AE version number. Is it 13.6.0.x?