I don't know if you ever got this resolved, if you did this may help someone else.
I just got a new toshiba netbook with a resolution up to 1024x600. I went to install a Visual Communicator 3 on it. I had the same problem when trying to start it where it wanted a resoluton on 1024 x 768 before it would work.
This is the work around:
1) right click on yoru desktop and select properties
2) click on the settings tab
3) Click on the advanced button (lower right area of the window)
4) Click on the Monitor tab
5) Deselect the checkbox labeled "Hide modes that this monitor cannot display"
6) Press OK
7) you should now be back at the Display Properties window and can use the scroller to change your resolution to 1024x768
Once you make this change, the screen will be bigger than your monitor. If you move the mouse to the top of the monitor the screen will scroll to the top, if you move it to the bottom the screen will scroll down.
After you are done with your video you can go and change the resolution back to 1024x600 if you like but I would suggest leaving that checkbox for hiding the modes unchecked so you have the extra resolutions whenever you need them.