The "queue" button sets up Adobe Media Encoder for this job and however many more that you wish to queue up.
Then, when you press Start Queue (in Adobe Media Encoder) these jobs run sequentially (not all at the same time).
It works great; don't study it any more, simply press the "queue" button and experiment!
I just can't seem to figure out how
That's why there is a Help file included with the program, so you don't have to "figure out how" to do things, you can just learn how to do them.
But...you do actually have to read the Help file for it to be helpful. Give it a go. There is a whole section on Exporting.
I'll recommend again that you start here to learn Premiere Pro.