No, there is nothing you are overlooking. Basically, Captions
are rectangular in nature. You simply insert them and arrange as
desired. If you wanted one rotated, you could basically take a
snapshot of it, pop it into an image editing application, rotate it
there, then insert it into the slide as an Image object.
If you strongly feel that the ability to rotate captions
should exist at some point, you should consider submitting a Wish
Form requesting it. You may visit the Wish Form by