Exporting as IDML is not only the method to allows the version of ID just previous to the one being presetly used to open a file, it's also used to clean out excess information that builds up in a file. If you still have the original file (i.e., you didn't write over it with the exported version), try doing a save-as (you will want to choose a different name or location, so as to not overwrite the original) and see how much it weighs.
I am having the same problem, only my file size has grown from 55 MB to 2 GB in two days, even though all I am doing is editing the text. My images are not embedded, so I don't know why this is happening... I hope you get an intelligent response to this problem.
I never did get an answer as to WHY this happens, but my experience and Michael's suggestion that exporting the file as IDML, then using that file seems to work.
It worked for me, too, and the file that had grown to 1,766,796 KB went back down to a very reasonable 35,872 KB.
The only (very slight) problem I encountered is that when I re-opened the file, the page numbering had shifted so that page 1 was page 302 (previously the last page in my file), so I had to adjust that.
Everything else seems fine, and I am very grateful.
The only thing I can find for why this happens is that "somehow" the file gets corrupted and that doing the IDML export fixes it.