I would think it would be read/write permissions for the design node beneath /etc/designs. Does setting that permission make a difference? I'm not really confident that the sidekick will handle it too gracefully though.
I had seen something about that a long time ago, but discounted it because there is nothing in etc/designs but css, js and some images. However, removing that permission (etc/designs/<my site>) does in fact remove the design mode button from the Sidekick. The thing you need to be mindful of is to then go back and add the read priviledges for the design elements that reside in etc/designs/<my site>
Yeah, the design stores the JS and CSS (sometimes in a Client Library), but it also stores the "design configuration" nodes. This what you set in design view. These nodes, which are separated per page component, are really what you'd want to remove permissions for. The JS and CSS assets should stay accessible.