16 Replies Latest reply: Aug 10, 2012 6:24 AM by sinious RSS

    how to make a desktop app ready for tablet

    birnerseff Community Member

      I have a flash drawing tool on desktop, and just tried to use it on a windows 7 tablet pc.

       

      The system comes with two virtual keyboards, a bigger one that the user has to actively open, and a smaller one that offers itself near many text entry fields (such as system file dialogs)

      Now with flash only the bigger one is available, and when I type into it, flash player seems to lose focus.

      What would be the best way to handle this situation?

       

      Another part of the story: would it be possible to have 3 checkboxes inside the flash app that reflect the state of shift, ctrl, and alt keys and send keypress / keyrelease events such that mouse events report correct keyboard state?

        • 1. Re: how to make a desktop app ready for tablet
          sinious MVP

          The keyboard is an application itself, so yes it will take focus.

           

          Your best option at this point, because a windows 7 tablet is not in the mobile device "realm", is to make your own keyboard. There is some integration with actual mobile devices to pop up specific mobile device keyboards but you can't take advantage of that because Microsoft's aim at tablets is making them "a real computer". So you don't have the mobile device keyboard functionality.

           

          You can find some readily made AS3 keyboards if you google around. Then you control look, feel and focus.

          • 2. Re: how to make a desktop app ready for tablet
            birnerseff Community Member

            Well, thanks a lot. I will go and look around for one

            • 3. Re: how to make a desktop app ready for tablet
              sinious MVP

              They're easy to find, hope you find one that meets your design needs. You're welcome and good luck!

              • 4. Re: how to make a desktop app ready for tablet
                birnerseff Community Member

                Hi sinious, while googling I had the impression that these things are not really "plugin".

                The app uses textfields with traditional focus, plus key event handlers, and also makes use of modifier keys on mouse events. While textfields could easily be extended to use the virtual keyboard, getting the other two things to work without excessive redisign would best be handled by the keyboard just dispatching key down and up events.

                Some preliminary testing indicates this does not really work. A "shift" checkbox that generates "shift key pressed" when checked does not influence the shiftKey property in mouse events

                • 5. Re: how to make a desktop app ready for tablet
                  sinious MVP

                  It depends on the keyboard and how it's implemented. Typically you just assign focus and then when you MouseEvent.CLICK a button the keyboard will send the clicked character to the focus. If shift/capslock/etc are pressed then they are toggles until the final button is pressed and then that key combination can be sent.

                   

                  Sophisticated keyboards can also simply generate events when keys are clicked to simulate keyboard input. If you can't find a freeware plugin to suit your needs, that would end up being the tedious part. Rewiring the keyboard so every character pressed emits the correct keyboard event.

                   

                  Demo Example

                   

                  Here is a simple CS4 saved down example of 2 clickable buttons and dispatching keyboard events as well as sniffing for them and doing something different based on what is received. Just the letters A and B with traces.

                  • 6. Re: how to make a desktop app ready for tablet
                    birnerseff Community Member

                    Hi,

                     

                    thanks a lot - but that is one part. The other one is this:

                     

                    shift and ctrl are checkbox-type elements

                    I have two listeners, one for the keyboard and one for mouse events:

                     

                    private function key(event:KeyboardEvent):void

                    {   trace(event);

                        if(event.keyCode == Keyboard.SHIFT)

                             if(event.type == KeyboardEvent.KEY_DOWN)

                                 shift.checked = true;

                             else

                                 shift.checked = false;

                        else if(event.keyCode == Keyboard.CONTROL)

                             if(event.type == KeyboardEvent.KEY_DOWN)

                                  ctrl.checked = true;

                             else

                                  ctrl.checked = false;

                    }

                    When I operate the shift key, the indicator lights up, and I see

                    [KeyboardEvent type="keyDown" bubbles=true cancelable=false eventPhase=2 charCode=0 keyCode=16 keyLocation=0 ctrlKey=false altKey=false shiftKey=true]

                    Subsequent mouse events show shiftKey true

                    Now there is also a listener on these indicators

                     

                    private function toggle(event:MouseEvent):void

                    {   var ck:Ckbox = event.target as Ckbox;

                        ck.checked = !ck.checked;

                        var kc:int;

                        if(ck == shift)

                            kc = Keyboard.SHIFT;

                        else if(ck == ctrl)

                            kc = Keyboard.CONTROL;

                       

                        stage.dispatchEvent(new KeyboardEvent(ck.checked ? KeyboardEvent.KEY_DOWN : KeyboardEvent.KEY_UP,  true, false, 0, kc, 0, ctrl.checked, false, shift.checked));

                    }

                     

                    So when I click the shift indicator, it generates an event

                    [KeyboardEvent type="keyDown" bubbles=false cancelable=false eventPhase=2 charCode=0 keyCode=16 keyLocation=0 ctrlKey=false altKey=false shiftKey=true]

                    However, the next mouse event says that shiftKey is false

                    • 7. Re: how to make a desktop app ready for tablet
                      sinious MVP

                      It's definitely in your hands to handle a context correctly. If you're using a mouse-based emulated keyboard then you only need to track if something is on or off, but not send that keyboard event for those keys. Keyboard events are probably best left only to keys that actually create a character.

                       

                      If it's shift, alt, etc then you should just be tracking the on/off state based on what the mouse clicks do and not send a keyboard event at all. When they finally press a key that produces a character, you look at your modifier keys (shift/alt/ctrl) and send the appropriate flags with the new KeyboardEvent. It's then your job to disable all the appropriate buttons again after the character is sent.

                      • 8. Re: how to make a desktop app ready for tablet
                        birnerseff Community Member

                        Hi,

                         

                        I have quite a few places in my code where mouse listeners record the state of keys, and there are also things like combo boxes. I had hoped that I could somehow add the virtual keyboard so that I do not have to modify every single mouse handler

                         

                        Maybe I should try to add high-priority mouse listeners to the stage and redispatch mouse events, with the states of virtual shift and ctrl keys added

                        • 9. Re: how to make a desktop app ready for tablet
                          sinious MVP

                          Context if tough to manage in a complex situation no doubt but it's the only sure fire way I see of you doing this while retaining focus. This is all about that, retaining focus. I've made a half dozen keyboards for various projects myself.. I've found no replacements for a custom looking kiosk/app.

                          • 10. Re: how to make a desktop app ready for tablet
                            birnerseff Community Member

                            Thanks a lot for your assistance, but I believe I will postpone that thing until I happen to find some ideas about the keyboard-mouse coupling. Keyboard input by itself does not seem to be a problem, but simulating shift / ctrl / alt keys for mouse (stylus) klicks is.

                            I will also have to check how important modifier keys are for flash components (i.e. whether one could live without multiple selections in a listbox)

                            • 11. Re: how to make a desktop app ready for tablet
                              birnerseff Community Member

                              Hi,

                               

                              I finally found some solution to this puzzle: The method is copied from mx.managers.Systemmanager:

                              register a high-priority listener for the mouse event. In the listener, if the event is not cancelable, it is the original event received from the player. Stop immediate propagation and then redispatch a new event, with cancelable = true (to distinguish it from the original event and avoid an infinite loop) and set key states according to the virtual keyboard

                               

                              I still have to find out whether to handle MOUSE_DOWN and CLICK, or just one of them

                              • 12. Re: how to make a desktop app ready for tablet
                                sinious MVP

                                Probably just one of them and it always feels snappier if you use MOUSE_DOWN instead of CLICK unless you plan on handling the situation where a user can press down on something but move their finger off the item to "cancel" that click.

                                 

                                Thanks for posting part of your solution. If you're all done please mark the thread as answered. Good luck!

                                • 13. Re: how to make a desktop app ready for tablet
                                  birnerseff Community Member

                                  Hi,

                                   

                                  I already have most mouse handlers on down event. The one thing I have to find out: if there really is a click handler that records modifier keys, would I have to redispatch the CLICK event as well, or would the DOWN be sufficient.

                                  • 14. Re: how to make a desktop app ready for tablet
                                    sinious MVP

                                    Both are the same thing, just an event. A modifier key event wouldn't do anything on the Flash level besides a true keyboard emitting the event, nothing more. It's up to you to decide if that event is important and do what you need with it. All you really care about is knowing the state of them when a character is added to the TextField so you know how to modify it so all you're interested in is enough events to make sure the character added has the proper modifiers added.

                                     

                                    If you're asking if there's a corresponding click handler when a mouse clicks an OS-level on-screen keyboard, no none is generated. Just the equivalent keyboard event fires.

                                    • 15. Re: how to make a desktop app ready for tablet
                                      birnerseff Community Member

                                      Hi,

                                       

                                      well, the virtual keyboard is no problem - all the mess was making the virtual modifier keys play nicely with a stylus.

                                      It turned out that redispatching MOUSE_DOWN looked reasonable but that redispatching both down and click events did not work at all.

                                      I did an estimate of how often there were click handlers used. Although at first glance most would handle simple UI elements and not use modifier keys at all, I should probably have had a look at all of these handlers.

                                      At that point I chose to dismiss the redispatch idea altogether and rather change all places in my code that refer to some MouseEvent.shiftKey to read

                                      e.shiftKey || vkbd.shiftKey

                                      simply because editing would be much faster

                                      • 16. Re: how to make a desktop app ready for tablet
                                        sinious MVP

                                        Always more than one way to skin a cat. Glad you got it working and good luck!