13 Replies Latest reply on Jul 24, 2015 5:11 AM by sinious

    Midi device input in Flash CC / Html Canvas

    Flabis66 Level 1

      Hi everybody,

       

      First of all, if it's not the right place for my question, please tell me where I can post questions about Flash CC/ actionscript 3.

       

      I'm building a small game (with Flash CC/ Html Canvas/ AS3 , on a Mac) for pianists to help them to play by ear. The app plays some notes (.wav or Mp3 files) and the user has to play them on a piano.
      Here an exemple of level 3. (Sorry it's a french version for now).

      For now, it works fine with the included virtual piano but as i want to add the possibility to play more than one note at a time  (ex: chords), i would like that the users use their midi keyboard as the input device.
      The only information I found on the web about midi input/output are too old and i haven't found anything about using this functionality with Flash.

       

      So, how to set a midi piano keyboard as the input device and collect the played notes?

       

      Can I use the WebMidiApi from W3org? If yes how?

       

      Thanks for any help or idea.

       

      F.

        • 1. Re: Midi device input in Flash CC / Html Canvas
          Flabis66 Level 1

          I think i found a solution but I need some help to implement it.

           

          I found some interesting information on the W3.org about the webmidiapi :http://www.w3.org/TR/2015/WD-webmidi-20150317/

          I've followed their instructions,  and wrote a short code in JavaScript:

          When i press a key on my midi piano it shows an alert box with the name(a number)  of the note. I would like to use these values (the notes names) in Flash.

          Here what i made for the moment

          http://flabis.com/testdev/index.html

          Now, how to integrate, or import, or translate or link the javascript code in ActionScript3 ?
          As I don't know if i have to import, or link, or if i can just copy the code, i don't know where to search.

           

          Thanks

          • 2. Re: Midi device input in Flash CC / Html Canvas
            sinious Most Valuable Participant

            To go this route, whip out Flash's trusty ExternalInterface class. It is the bridge between Flash and JavaScript (and any other hosting app for that matter). There's simple example code on the bottom of the ExternalInterface class page itself:

            ExternalInterface - Adobe ActionScript® 3 (AS3 ) API Reference

             

            Using the sample I'm sure you can easily get your values from AS3<->JS. This used to be called FSCommand if you've been using Flash for a long time. There's a million tutorials of all kinds on this if the source code in the class page isn't enough for you but it's very easy to understand.

            • 3. Re: Midi device input in Flash CC / Html Canvas
              Flabis66 Level 1

              Thanks a million. Now I know where to look and can continue my work.

              • 4. Re: Midi device input in Flash CC / Html Canvas
                sinious Most Valuable Participant

                You're welcome and good luck!

                • 5. Re: Midi device input in Flash CC / Html Canvas
                  Flabis66 Level 1

                  Sorry I'm back

                  I've tried with ExternalInterface, but each time I test my project, the html file is replaced by a new one and I have to re-write (or copy/paste) manually my functions ( see code below) in the html doc.

                  I can save this code in an independent .js file but I still need the reedit the html file to add the link to my .js file.

                  Is there any other way just to run the next code in actionscript ? I would like attach the next code to a button on my canvas instead of the window load event and use the values "event.data[1]" in flash instead of the alert box.

                  Thanks for any help.

                  // here is the code from W3org from where i delete the useless lines
                  var context=null;   // the Web Audio "context" object

                  var midiAccess=null;  // the MIDIAccess object.

                  window.addEventListener('load', function() {

                        // patch up prefixes

                        window.AudioContext=window.AudioContext||window.webkitAudioContext;

                       context = new AudioContext();

                       if (navigator.requestMIDIAccess)

                          navigator.requestMIDIAccess().then( onMIDIInit);

                  } );
                  function onMIDIInit(midi) {
                          midiAccess = midi;
                     var haveAtLeastOneDevice=false;
                          var inputs=midiAccess.inputs.values();
                          for ( var input = inputs.next(); input && !input.done; input = inputs.next()) {
                               input.value.onmidimessage = MIDIMessageEventHandler;
                               haveAtLeastOneDevice = true;

                                 }

                           }

                  function MIDIMessageEventHandler(event) {
                          switch (event.data[0] & 0xf0) {
                        case 0x90:
                                 alert(event.data[1]); // shows the number of the note (60 = C, 62 = D, 64 = E...)
                           // now i want to use theses values inside Flash
                     }

                      }

                  • 6. Re: Midi device input in Flash CC / Html Canvas
                    sinious Most Valuable Participant

                    Your best bet is to go in publish settings and uncheck the HTML output box so it doesn't overwrite the HTML anymore. Put your code in the HTML file and keep it loaded in a browser. Just hit refresh every time you export the SWF and check your changes.

                     

                    Otherwise you can safely assume ExternalInterface is going to work. You can 'mock' the data, as in hard code the data and pretend it came in via JavaScript. Just test using that and hook it up live later.

                    • 7. Re: Midi device input in Flash CC / Html Canvas
                      Flabis66 Level 1

                      good idea !! I'll try that.

                      • 8. Re: Midi device input in Flash CC / Html Canvas
                        Flabis66 Level 1

                        It works!!

                         

                        I attempted with ExternalInterface but i was getting a lot of errors.
                        Looking forward for an exemple using this method, i found another solution more simple. So simple that I don't know why i didn't try it before.

                         

                        I just need to add 2 lines on my html page:


                        playednote = event.data[1] // this is the value of the played note on the midi keyboard This line replaces the alert one.

                        document.myFunction(playednote); // myFunction is a function in Flash/AS

                         

                        Your last tip about unchecking the HTML ouput is very usefull !! I win a lot of time.


                        Thanks for your help, now i can continue. I'm happy

                         

                        Here is the script in my html page:

                        <script>

                        var canvas, stage, exportRoot;

                         

                         

                        function init() {

                          canvas = document.getElementById("canvas");

                          exportRoot = new lib.Sansnom3();

                         

                         

                          stage = new createjs.Stage(canvas);

                          stage.addChild(exportRoot);

                          stage.update();

                         

                         

                          createjs.Ticker.setFPS(lib.properties.fps);

                          createjs.Ticker.addEventListener("tick", stage);

                        }

                        var context=null;   // the Web Audio "context" object

                            var midiAccess=null;  // the MIDIAccess object.

                            var playednote = "";

                            window.addEventListener('load', function() {

                              // patch up prefixes

                              window.AudioContext=window.AudioContext||window.webkitAudioContext;

                         

                         

                              context = new AudioContext();

                              if (navigator.requestMIDIAccess)

                                navigator.requestMIDIAccess().then( onMIDIInit);

                            

                            } );

                         

                         

                            function onMIDIInit(midi) {

                              midiAccess = midi;

                         

                         

                              var haveAtLeastOneDevice=false;

                              var inputs=midiAccess.inputs.values();

                              for ( var input = inputs.next(); input && !input.done; input = inputs.next()) {

                                input.value.onmidimessage = MIDIMessageEventHandler;

                                haveAtLeastOneDevice = true;

                                }

                             }

                         

                         

                           function MIDIMessageEventHandler(event) {

                              switch (event.data[0] & 0xf0) {

                            case 0x90:

                                 

                                  playednote = event.data[1]

                                  document.myFunction(playednote); // myFunction is a function in Flash/AS

                              }

                            }

                        </script>

                        • 9. Re: Midi device input in Flash CC / Html Canvas
                          sinious Most Valuable Participant

                          The errors you were getting are most likely from one or all of these causes:

                           

                          • Not waiting in JS until the page is finished loading
                          • Not running it from a HTTP server or from alternate domains without specifying the script access or telling Flash to allow the domain.
                          • Not specifying the callback in Flash for EI

                           

                          Here is a very simple HTML / JS + AS3 example:

                          http://www.ertp.com/tmp/ExternalInterfaceComm.zip

                           

                          The important ingredient here is I didn't just run the HTML file off the filesystem. That has a whole layer of complexity I don't want to get into if you're running this from a server. It waits until document load and then a couple extra seconds, while firing off the function call.

                           

                          Simple HTML:

                          <!DOCTYPE html>

                          <html>

                            <head>

                            <meta charset="UTF-8">

                            <title>ExternalInterfaceComm</title>

                            </head>

                            <body>

                            <object type="application/x-shockwave-flash" data="ExternalInterfaceComm.swf" width="550" height="400" id="ExternalInterfaceCommEx" name="ExternalInterfaceCommEx">

                            <param name="movie" value="ExternalInterfaceComm.swf" />

                            <param name="allowScriptAccess" value="always" />

                            </object>

                           

                            <script type="text/javascript">

                            // wait until page is loaded, then:

                            addEventListener('load',function(){

                           

                          // send "Hello, World." to Flash, to the function named

                            // "myFunctionInFlash", which Flash is listening for.

                           

                            // delay by 2 seconds

                            setTimeout(function() {

                            document.getElementById('ExternalInterfaceCommEx').myFunctionInFlash("Hello, World.");

                            }, 2000);

                           

                            });

                            </script>

                            </body>

                          </html>

                           

                          In AS3, Flash is told the callback to add via the name I use in JS and I tell Flash to allow the request from any domain:

                           

                          import flash.external.ExternalInterface;

                           

                          // stop the frame loop

                          stop();

                           

                          // allow any domain

                          Security.allowDomain('*');

                           

                          // add the JS listener

                          ExternalInterface.addCallback("myFunctionInFlash", receivedFromJavaScript);

                           

                          // when JS runs thisFunctionInFlash(), it maps to this function

                          function receivedFromJavaScript(value:String):void {

                            status.appendText('Received from JS: ' + value + "\n");

                          }

                           

                          I have a dynamic TextField on stage with the instance name "status" to output the text to. All together, after 2 seconds, run from a local HTTP server:

                           

                          EI.png

                          • 10. Re: Midi device input in Flash CC / Html Canvas
                            Flabis66 Level 1

                            Thanks for your time.

                            Your method works fine in Actionscript when i create a swf file but not when i'm in an HTML5 Canvas.
                            (I always make my tests on a server and the page is completely loaded before i press a key.)

                             

                            What i've learned today:

                            In Html5 Canvas, we don't need the ExternalInterface because everything stands in 2 files : the .html and .js.
                            It's only one interface, there isn't any external interface like a .swf file embed in a html document.

                            So we can pass values from one function to another exactly like we do inside actionscript (or inside a javascript)

                            We just need to add "document." before the function name when we call it and it works fine.

                             

                             

                            document.myFunction(myValue);   // inside <script> tag

                             

                            and in AS:

                             

                            document.myFunction = function(v) {

                              // my code here;

                            }

                             

                            Simple, no?

                             

                            Ok, now my problem is resolved, i'm going to the next one : How to manage the sounds in Html5. This will be for another post if i can't find a solution before tomorrow

                             

                            F.

                            • 11. Re: Midi device input in Flash CC / Html Canvas
                              sinious Most Valuable Participant

                              I somehow thought you were using Flash as you mentioned ActionScript. My mistake.

                               

                              Canvas projects do not use ActionScript, it is 100% JavaScript. Just be aware of that when you are in the code editor inside Flash. You may put all of your JS inside there.

                               

                              Do be aware that the 'scope' of the Flash document is of importance however. Otherwise feel free to fully utilize JavaScript, including the MIDI methods, inside the code you write inside Flash.

                              • 12. Re: Midi device input in Flash CC / Html Canvas
                                Flabis66 Level 1

                                Yes, I'm using Flash CC professional , opening an Html5 Canvas and there i see exactly the same thing as i open an actionscript project: a scene, a timeline, an "Actions tab" ...
                                The syntax used in the Actions panel is the same as in ActionScript. The JS syntax does not work.The only difference is when publishing the project, instead of creating a .swf and .html, it creates a .js ans a .html.
                                Some features are removed or greyed-out if there are not compatible with HTML5 Canvas.

                                I though it was actionscript3. Isn't it?

                                 

                                 

                                captureflash.jpg

                                • 13. Re: Midi device input in Flash CC / Html Canvas
                                  sinious Most Valuable Participant

                                  Nope, that is a combination of JavaScript and the CreateJS framework. To prove it go ahead and put JavaScript in there. Stick an alert() or console.log/info in there, which you know is JS and doesn't work in AS.

                                   

                                  Yes, they didn't change the tab name to JS or Code but I assure you it is pure JavaScript. Look in the .js file it makes and you will see your exact code, wrapped by CreateJS. No transcode from AS to JS. Otherwise you wouldn't need to preface objects with 'this' or 'that'. So feel free to write JS in there like you would a .js file. Just understand if you desire global scope to preface with 'window'. E.g. window.myGlobalFunc(); and 'document' accesses the DOM just as you think E.g. document.getElementById('someId')...