So my question is why Adobe does not care to make those rather basic improvements.
They have priorities, and obviously, these are different than your priorities. Adobe's priorities must be good enough for most people, as Lightroom is an extremely popular program
Regarding your 2nd bullet item, this hierarchical list is displayed and does allow drag and drop, and it has been this way since I started using Lightroom when Lightroom 2 was the current version.
Regarding your 1st bullet, I have never had any problem using keywords to filter my photos, and I use them extensively when I search for specific photos. Removal of specific metadata at export is possible by a plug-in, and individual keywords can be designated as not exportable right in Lightroom (no plug-in required)
* People, places, etc are all stored in flat keywords. This is a security/privacy issue as names cannot be easily removed for sharing without affecting other tags. Further, it is difficult to select and sort pictures according to different criteria.
Well, compared to iView/ExMedia/MediaPro, Lightroom allows you to set any individual keyword so it is private and does not export. Sorting may be better in iView/ExMedia/MediaPro but it doesn't have anything like Lightroom's smart collections which allow quite sophisticated selection.
* Defining keywords is limited to 9 items. That is WWAAYY too small. This should be a (hierarchical) list that is displayed in its entirety allowing drag & drop.
There is no equivalent of the 9 item panel in iView/ExMedia/MediaPro, which is a way of entering frequently-used keywords. LR has a hierarchical list which is similar to the list in iView/ExMedia/MediaPro and allows drag and drop.
* There is no tool to transition from the biggest competitor to Adobe (there are several incompatibilities but it would be relatively easy to translate them)
The biggest competitor isn't iView/ExMedia/MediaPro. Until recently it was probably Aperture, and Adobe have now introduced a tool to assist transition.
So my question is why Adobe does not care to make those rather basic improvements. This is not just my reason for not switching from PhaseOne / MediaPro but MANY other people in that industry. Adding image manipulations into PS is silly as people who care about archiving/cataloging/managing images through Lightroom probably own Photoshop Light or the full-blown PS.
Many knowledgeable iView advocates such as myself or Peter Krogh did move to Lightroom, and part of the reason was that Aperture and then Lightroom demonstrated the value of combining image adjustment with cataloguing. There are certainly some iView features (vocabulary files, list view, people panel, custom fields) which I'd like in Lightroom, but none of the ones you mention! I suspect you need to spend longer with Lightroom.