What is going on is that the Adobe Paragraph Composer is trying to "optimally" re-rag your entire paragraph. If you become accustomed to its assumptions, you may well find that it is incredibly useful - but if you're not used to it, if you're used to other less-intelligent paragraph composition methods (PageMaker/Quark/Word/???) then it won't "inspire confidence."
You can change it to the Adobe Single-Line Composer in more than one place - look for it in the Paragraph panel flyout menu.
Also, I would advise that you should reconsider using a manual line break (shift-enter) to re-rag your paragraphs. A no-break space (alt-ctrl-X on Windows, found under Type -> Insert Whitespace) or applying No Break (found in the flyout menu on the toolbar, as well as a few other places) are going to work much better with any of the composers available in InDesign.
Also not sure why "dates" was not at the end of the second line to begin with as there was room.
I don't always agree with the decisions that the Paragraph Composer makes, but it probably "thought" that the whole paragraph would have a better rag - or fewer rivers, perhaps? When I first started setting large volumes of text in ID using the Paragraph Composer, I felt the same way as you do. What I did then is: I timed myself. I was laying out the same handbook in eight Latin-script languages, and I found that using the Single-Line Composer I was spending more than 20% more time on getting a pleasing rag. It's true that using the Paragraph Composer felt like I was setting type while wearing mittens, at least at first. But once I figured out the tolerances of the Paragraph Composer (and how to push it around using the Justification dialog and judicious use of No Break and non-breaking spaces), I found that not only did I like the results better, but that I was more than 30% faster in the end.
It still feels like I'm wearing mittens, though.
I am not sure I would consider it a less-intelligent situation, although I know what you are trying to say. I don't consider Auto Correct (or the like) to be more intelligent because the program does it for you. Don't get me wrong, I am not a technophobe. I think there are great time time saving aspects to most software, just not sure that something like this is not taking too much control away from the designer. In my particular example, I would have been quite happy if I could have somehow had "dates" at the end of the second line. I could only get it to go up by dropping "status" to a fourth line which then created an unnatural space at the end of the third.
I will have a look at Single-Line Composer either in individual situations when it isn't working or universally, if that is possible.
I am not sure why I would use shift-enter as that would send the text after to the next text box as opposed to dropping a word to the next line with a soft return (shift-return).
I could only get it to go up by dropping "status" to a fourth line which then created an unnatural space at the end of the third.
In a situation like that I usually try using a non-breaking space between the current last word and the one I want to bring up to see what happens. Sometimes the last word moves to the next line, but often the one you want moves up.
Oh are you a Mac user? I often forget that Enter and Return are different. I meant "manual line break, however it is that you produce it with your keyboard or mouse or whatever." In general I always advise against the use of the manual line break to control the rag or composition of a paragraph. I've spent a considerable chunk of time removing them in order to produce a document that will e.g. easily make the transition to HTML or ePub or content-management system or translation memory database or whatever. The point is that a manual line break is for me a less repurpose-able, more "final" method of affecting wrap than is something that prevents a linebreak, like a nonbreaking space. These considerations may not apply in your case, but I'm sure that they apply at least some to the sixty-odd people who have already read this thread, which is why I'm soapboxing about it.
Control is only being taken away from you if you are unaware of your options - I switch from the Paragraph Composer to the Single-Line Composer sometimes, myself. I have a set of scripts to which I've assigned keyboard shortcuts so I can switch composers on the fly, which I do all the time (mostly from Adobe Paragraph Composer to World-Ready Paragraph Composer). I turn off auto-correct on almost every single application I use, usually because it hampers me instead of helping. But this is a case where, if you care about good paragraph composition (and you obviously do!) you should consider it.