First, have you ensured that your Premiere Elements project is set up for DSLR 24p? (You don't say what resolution the video is, but your project specs should match that also.)
That said, if you can't change the audio speed without changing the pitch, why not use Time Stretch to change the video speed?
Although I am concerned that changing the audio speed changed the pitch. How out of sync are these files!?
Well the video files are 1080p 24fps, I did change the audio speed to 100.25 and that worked but only for a bit. it seems like I can change the speed of the audio to match shorter clips but after 5 minutes it starts to drift by 1 frame, which is better than before, the audio was slower by 5-10 frames 5 minutes in. I think the audio speed is 25? but again I don't really know.
But which project preset did you select for your Premiere Elements project?
Also, what is the sampling rate of your audio? If you're using a 12-bit 44,100 WAV file, you should be fine. Other sampling rates could cause you problems.
If your audio is another bit or sampling rate, you can easily convert it to a compatible format (that will likely resolve your issue) with the free download Audacity.
Well I'm using the 24p 1080 project preset. and the audio is 32-bit 44,100 wma file, I imported it to try and convert it to 12-bit but it seems like audacity only does as low as 16 bit. That likely is the problem.
The 16 bit should work.