I don't think you're missing anything, that's just one (of several) features that are in Bridge, but not in Mini Bridge.
But given that Bridge is one button click away (you'll find the button on the Application bar in InDesign or the Bridge button at the top of Mini Bridge), I don't see what's the big deal.
Bummer! I was hoping to get it all done in mini Bridge. It didn't make
sense to me that I'm able to add files in mini Bridge but not delete.
I think, in a lot of workflows, MiniBridge is a tool that aids in the placing of content but not really the organization or changing of it. That's why features like renaming and sorting are so weak, compared to other tools. As a result, they don't want to make it too easy to delete content, because that might put some users off ("I had a bad experience with Mini Bridge. The Mini Troll ate my Mini Snippet.").
I'd agree with John. I think Mini Bridge has the primary job of viewing and placing content into InDesign. Use Bridge to manage content.