Lucky you! We have reading and viewing material that 'splains it all!
After Effects Help / Previewing
I think Dave's first link will answer your immediate question. Because I'm guessing you're pressing spacebar to view the videos in AE. After Effects is not video editing software. It doesn't play back video like video editing software does because it treats video in an entirely different way.
Dave's third link is, as he says, the most valuable of them all. It will help you get past this and many, many other problems you will run into if you don't take the time to get a solid foundation in how AE works.