Alas, the PSE slideshow isn't that sophisticated. You get one track per slideshow. If you also have Premiere Elements that may let you do something more elaborate. (Don't use PRE, so I can't say for sure.)
You can combine the tracks in another program before adding it to the slideshow and then adjust the timing for the slides and transitions to make it work out right, but that's about as far as it goes in PSE.
How would depend on the program. There are many audio editing programs out there. Audacity is free and quite popular:
Most wil have a bit of a learning curve at first.