What do you mean by "catalogues" ?
In Lr speak the "catalog" is the data base where Lr stores / saves everything you do in Lr.
Or do you mean images?
Single images or a group of selected images are removed from Lr by right-clicking and selecting <Delete Photo>. You are then given the choice of <Deleting from disk> which sends your image(s) to the Recycle Bin / Trash Can or of <Remove> which removes the image from Lr but leaves it on the hard drive wherever it is right now.
You also can remove folders from Lr by right-clicking them and selecting <Remove>.
When you are in a Collection you can only remove an image from the collection; you have to go to the folder first to do the above mentioned things.
If you really want to delete all edits, etc. contained in the catalog, then delete the folder containing the .lrcat file - that'll remove it from disk, permanently (or move it to recycle bin). Dunno how to remove it from the list of recently edited catalogs. Note: catalog must not be open in Lightroom.
*** WARNING: Only do it if you are sure you never want to use the catalog again.
So my question is this? Once I remove the catalog the way you discribed this process, when I open up LR4 the Catalog should not be there any more right?
When Lr does not find any catalog it will come up blank and wants to import images. If you then cancel import Lr will show/display no images at all.