Which H.264 CODEC do you have installed? There are three popular ones: Apple's (installed with the later versions of QT Player), Lead's and MainConcept's. Many users report much better results with the Lead, or the MainConcept.
PS - Welcome to the forum!
It also makes a big difference where this MPEG4 comes from.
Video from pocket camcorders, like the Flip, can not be used in Premiere Elements without a conversion to a more standard format.