Just delete it from the response table. Once you do that its deleted from our servers.
There was a typo in my question! It should have said, how do I make sure I don't lose my data on form central! Oops, so opposite from what I wrote :-)
Ah, you can export to Excel or make Duplicates of the form file that include the responses.