I'm afraid that option to create folder-like sets for swatches doesn't exist in Photoshop. But there're other options:
1) Add the swatches you want, to your Swatches folder. No need to delete the original colors to create the set you want. After adding your color set, use the panel menu to get to the Preset Manager. In the Preset Manager, you can shift-click the colors you want to save to a new set and then hit the button for Save Set. The default location is with your Swatches, so to get to them, you'd always find them in the Swatches drop-down list, designated under User Sets. If you're doing it for one client or job, you may want to save the set to the client or job's folder. You can always retrieve it by using the drop down menu for Replace Swatches. Select Replace, instead of Append, and there they are.
2) You can create a new cloud set and individually add swatches to that online set - from there, you can use swatches from there and add them to your Swatches panel. I just hate that you have to single-drag them TO the library. An alternative is that when you save a swatch, CHOOSE "Add to my current library"
3) I don't think there's any way to do the subsets you request in Photoshop. For me, I'd create them and use the Preset Manager for each.
4) I tried deleting the swatches history. I know you can delete swatches one at a time using your Option/Alt key and clicking them, so I tried it on my history. It LOOKED like it was working, but I have SUCH a long history, I'd have to do that for hours to see if it's really doing anything. There's no quick way to do it though, as far as I know. Perhaps someone else on here has that answer.
I'll include a short video of my option #1, since that's the most intensive, but IMO, the most useful answer for you.