Probably the best way to do, what you describe, is with the Project Archiver. Direct that to create your "archived" Project onto the external HDD. I always recommend testing that archived Project, to make sure that it functions perfectly, before touching the original Project.
Finally, to Delete your original Project, after you have transferred it to the external HDD, whether through Project Archiver, or via Windows Explorer, your actions, and the work associated with that process, will depend on how you set up your Project initially. If you follow the steps in this article: http://forums.adobe.com/message/5681692#5681692 , then Deletion is simple - just Delete the Project's "root folder," Deleting all of the sub-folders in that process.
If you have Assets scattered all over your system, in different folder, or on different HDD's, you will need to Delete those individually.
What version of Premiere Elements are you using and on what computer operating system is it running?
Depending on the circumstances yet to be exchanged, the easy way out would seem to be to archive the existing project via the program's Project Archiver/Copy Option with the Copied Folder pointed to the external hard drive. That folder will contain the project file (project.prel) and its assets including copies of source media that went into the project. Then you can delete the project file, its assets, and source media from the Local Disc C. DO NOT DELETE SOURCE MEDIA ON LOCAL DISC C THAT ARE ALSO BEING USED IN ANOTHER ACTIVE PROJECT.
After that you can work with that particular program from that location.
Before you start a new project, you can set the project Save In: (new project dialog) and program assets (including Scratch Disks) to the external hard drive, best to its own folder.
In this type of situation, the prime concern to to avoid media disconnect issues. If you move or delete source media that went into a project after the project's save/close, you are going to have major problems with "Where is file....?" and the like.
Add On...I did not see Hunt's post(s) until after I posted mine.
Yes, digital data creates identical copy's of itself.
When transferring files/folders you have 2-choices...Move and Copy.
Move is faster.
Copy is safer.
(should you suffer a power failure)
I use a Thermaltake external dock.
This allows me to have SATA HDD's and SATA SSD
as defined folder drives.
I use a specific HDD for projects.
The HDD will contain 2-folders.
Un-edited and edited.
Any files standing without a folder can be deleted.
Once the project is completed--I decide about the 2-folders.
(alone there on a HDD)