I am thinking about purchasing a Creative Cloud membership to upgrade from Elements but don't want to do it if I don't have basic editing capabilities for videos.
If you buy full Creative Cloud membership plan it includes video editing software in it like Premiere Pro.
If you specifically want to use video editing software you can also buy single app membership of Premiere Pro
You can refer for plans and pricing : Creative Cloud pricing and membership plans | Adobe Creative Cloud
Hope it helps.