Are you sure you're not looking at the Elements Organizer?
Premiere Elements will only include the media files you've added to your project.
Which version of the program are you using and which panel shows all of your media? Pre-version 10 versions of Premiere Elements displayed the Organizer's media list.
I am using Premeire Elements 7.0 and it uses the libraries established in Photoshop 7.0 as a media sourse. You cannot seem to bypass it wanting to load all the photos. You can filter using the keywords and albums but you cannot seem to point to a blank folder as a default.
As I said, under the Organizer tab in version 7 is a display of all of the media files in your Organizer's catalog -- every file you've edited in Photoshop Elements and potentially every media file on your computer. (As you add files to your project, it will display on the most recently added.)
These are NOT the files added to your project! The media added to your project are displayed under the Edit tab wehn you click the Project button.
These very confusing file displays were re-thought in version 10, in which only the files actually added to your video project are displayed in the program.
As Steve points out, there are really three programs in the Premiere and Photoshop Elements "suite," PrE, PSE and then Organizer, which links between the two, and is an Asset cataloging program. Because of the commonality of the GUI (Graphic User Interface), there can be confusion as to exactly which program one is in.
If you launch just PrE, and not Organizer (can be launched from within either PrE, or PSE), the only Assets/media that you will see is that, which you have Imported into your Project.
Also, having the Assets for a Project, in a separate folder, or folder structure, is a great way to set up the Project. This article goes into details on one set up method: http://forums.adobe.com/message/4491450#4491450
Good luck, and hope that helps,