Unfortunately, you cannot turn of CS5 Frame Edge Highlighting in InDesign. There have been several threads about this in the non-scripting forums, including the locked 201-post monster, Frame edge highlighting, and the feature request Allow Frame Edge Highlighting to be turned off, and frame edge highlights on hover.
Basically the takeaway is that it only happens with the selection (V) and direct selection (A) tools, and that multiple people have asked Adobe for a control to turn it off, and if it matters to you, then you should too, by filling out the wishform. (I'm actually surprised/disappointed we haven't heard from anyone at Adobe that turning this feature off is slated for 7.0.4, or that nobody has produced a plugin or patch to turn it off.)
With respect to your script, it changes the View > Show Frame Edges setting in the InDesign application, not in the current document. So you see nothing. If you add "tell current document" (or whatever the right AppleScript is), then you'll see it turns off frame edges. But that doesn't affect frame edge highlighting .
Oh, whoops! I forgot the best workaround. From Mike Rankin in the InDesign Magazine Tip of the Week in January.
Basically, just keep your hand on the spacebar.
TIP: Spring-Loaded Tools to the Rescue
InDesign tries very hard to help you align objects when you drag by snapping them Do you dislike the Frame Edge Highlighting that appears when you mouse over frames in CS5? Spring-loaded tools can help. Frame edge highlighting only appears when you use one of the selection tools. With spring-loaded tools, you can press and hold single keys to temporarily switch to any other tool. As long as you hold the key for more than a second, you'll switch back to your previous tool when you release the key. You have plenty of choices (t, \, p, f, m, n, c, e, g, z, etc).
Holding the spacebar also prevents the highlighting from appearing, and it's a nice big place for your fingers to rest. Heck, you don't even have to pay much attention to where you press--just rest your hand in the middle of the keyboard and lift it up when you want to select something. But don't hold v or a, which are the shortcuts for the selection tools.
- Mike Rankin
(Though in fairness, Mike had suggests this several times in the past year, though not with the spacebar. E.g. he recommended 'H' back in June over on InDesignSecrets. "H" for Hate.)