Are you by any chance trying to record on a VM?
Elaine, yes i'm using VM. So this is the reason? Is there any solution rather then try screen capture via host machine?
As far as I know, there isn't a workaround. You'll notice that other screen recording apps (like QuickTime) also fail, so it seems to be an overall limitation with the VM itself. Sorry to be the bearer of bad news, but hopefully recording from the host OS will work for you.