In my books, I recommend that, whenever you start a new project, you store the project file in a new directory folder. That way clean-up is as simple as just deleting that folder.
Otherwise, you'll have to do some digging.
Go to the folder in which your .prel file is saved and look for the following folders:
Adobe Premiere Elements Auto-Save
Adobe Premiere Elements Preview
Browse into each of these folders and delete the directory folders and/or files that have the same name as your .prel file.
I found this whilst looking to an answer to a similar question about removing files once you've finished with a project, and wanted to make a comment for any future finders...
Even if you save your project media and proj files in a completely separate folder, from what I can see Premiere will still save GB of files elsewhere. NOT in the same folder.
I have just found 80GB of files (which on an SSD is quite a bit), sitting in C:\Users\Dave\Documents\Adobe\Premiere Pro\10.0\Adobe Premiere Pro Video Previews
and i came here searching for whether it's safe to just delete these...
I'd expect so. Although those files are for Premiere Pro, not Premiere Elements.