9 Replies Latest reply on Jun 14, 2014 9:31 PM by ChipNow

    audio control and sync on mobile

    ChipNow Level 1

      I am building a multi-section, web-based, interactive presentation.

      As part of testing architecture, I have two symbols, each with audio. The symbols do not overlap on the main timeline. No symbol, stage included, has autoplay enabled. Desktop browsers work as expected: after a tap on an object on the main timeline, the symbols play, with audio, in their timeline order. Mobile doesn't work as expected: only the audio of the first symbol plays. Whichever symbol appears first in the timeline gets its audio played. The second symbol's audio does not play, even after changing the symbol order and adding a sym.stop() and another tap after the first symbol finishes.

      Should I try to get a symbol to assign itself to whatever audio channel the mobile browser has configured for the first symbol? From the "Mobile Considerations" section on this page (bottom), it sounds like this reassignment would happen automatically. Am I configuring something incorrectly?

      In a related matter, how do I make sure the audio is synced with the animation?


      Here is a link to an example.

        • 1. Re: audio control and sync on mobile
          ChipNow Level 1

          So - I've answered my own question. In short, audio on mobile is painful. AFAICT, it is limited to a single tag which needs to be created (new Audio()) in compositionReady and the source changed whenever another source is needed. My problem now is how to deal with the different MIME types for that Audio() object. Anyone know?

          1 person found this helpful
          • 2. Re: audio control and sync on mobile
            Preran Adobe Employee

            Hi ChipNow,


            Thank you for posting your solution. It will help other users that come across this post.




            • 3. Re: audio control and sync on mobile
              ChipNow Level 1

              If I understand what I am seeing, it looks like someone figured out how to use multiple <audio> tags on mobile:


              LeanBack Player | Multiple HTML5 <audio> elements example


              Works on iOS and Android. They play simultaneously. Anyone have any "best practice" thoughts on how to integrate this capability into Animate? This would be a great addition to Animate's native capability - again, if I understand the page source from the example above.


              Your thoughts are appreciated.

              • 4. Re: audio control and sync on mobile
                jbowden Adobe Employee

                Hi Chip,


                Not sure I'm following your thread…are you maybe referring to the Web Audio API? That's only supported in Chrome (partial support in Safari). Animate and all HTML 5 browsers support the HTML 5 Audio element, which is different than the Web Audio API.


                To get back to the topic - audio elements on mobile devices require an explicit command to play audio, and that command must come from a user event such as click, touch, or swipe. The syntax is like this:




                If your audio is within a symbol, then you'd get the symbol first, i.e.:




                The idea when authoring for mobile would be to issue both the symbol play command and the audio play command simultaneously from a user event, like so:





                Hope this helps a bit,


                1 person found this helpful
                • 5. Re: audio control and sync on mobile
                  ChipNow Level 1

                  Hi, Joe.


                  I am confused. I have tested LeanBackPlayer (examples above), on the following browsers:


                  Mac: Safari, Chrome, Firefox, Sleipnir, SeaMonkey and Opera

                  Windows: IE, Firefox, Chrome, Sleipnir, SeaMonkey and Opera

                  Android: Dophin, Chrome, Firefox, Opera, Sleipnir, and UC

                  iOS: Safari, Chrome, Dolphin, Mercury, Atomic, UC, Opera Mini, Sleipnir, and Opera Coast


                  Only the browsers in bold had to use the file download fallback. All others played the multiple HTML5 audio tags simultaneously. The fact that it runs in IE seems to rule out any use of Web Audio.


                  The point is that more than one <audio> tag can be played on mobile, contrary to popular belief. See this post. Animate just acts as if it can't. It seems only to allow the first <audio> tag played to play subsequently. I don't know why.


                  I understand the theory your commands propose - though you don't mention the multiple tags that are at the heart of this issue. I have tried multiple ways of starting sound without relying on autoplay. It doesn't seem to work in practice on mobile with multiple symbols or multiple audio elements in the same symbol. The first audio element plays, no trouble, whether in different symbols or in the same symbol. Any subsequent .play() calls, whether via a trigger or event, to other audio elements, yields nothing but silence on mobile. The only thing I have been able to get to work in Animate on mobile is to create a new Audio() object in creationComplete


                       var adobesound = new Audio();



                  and change the source when I need to when I need to play a different track


                       var theSoundtrack = sym.getComposition().getStage().getVariable("compositionSoundtrack");

                       theSoundtrack.src = "media/mySound.mp3"; //  what do I do if there are multiple <source> tags for MIME options?



                  While this works on mobile, it's a pain for a number of reasons, not the least of which I don't yet know how to deal with MIMEtype issues for such assignments. And this essentially bypasses Animate's sound facilities.


                  Anyway, from your responses, I must be missing something very basic. Do you have an example with multiple audio elements that works on mobile that I could look at?



                  • 6. Re: audio control and sync on mobile
                    jbowden Adobe Employee

                    Hi Chip,


                    I'm not saying that it's not possible play multiple audio files (simultaneously or not) in iOS - simultaneous audio playback has been possible since iOS 6. But each of those play commands must come from a user event and be explicit to the audio element itself, and not to a timeline containing the audio element.


                    I'll post an example shortly based upon on your example, which will play correctly on iOS/Android - but through user interaction.




                    • 7. Re: audio control and sync on mobile
                      ChipNow Level 1

                      Thanks, Joe.


                      So as many audio elements as I have, I have to have that many interactions? Thanks, Apple. I'm sure that will change when advertising or data rates change.


                      What confuses me is that when I start the timeline on the stage, the interaction calls a sym.play() on that timeline, not on the audio element - and it works. I guess you said I was lucky to have one play. At very least the execution isn't consistent.


                      What a nightmare.


                      Do you know if there is a maximum file size for audio? Maybe I should just do the audio equivalent of a sprite.


                      What do you think about reassigning the <source> tags of the one audio element that got clearance from a single interaction?

                      • 8. Re: audio control and sync on mobile
                        ChipNow Level 1

                        By the way, written at the announcement of Animate 3, Sarah Hunt's article states specifically that iOS cannot handle more than one audio stream. I didn't question the assertion and, all this time, I thought this restriction was central to the issue I was having on mobile. Thank you for confirming I've been barking up the wrong tree. I can move on.


                        Multiple user interactions are not going to work for my application. Don't worry about creating an example. Thank you for your willingness.


                        It seems I have two options:


                        1. Reassign the source of a single tag that was "activated" by a single user interaction. Pro: a number of smaller audio files that are less likely to get out of sync and can be downloaded sequentially. Con: not as easily aligned with the timeline in Animate.
                        2. Do an audio sprite with one large file. Pro: no source switching and easily managed in Animate. Con: size - with all its implications for download, maintenance, and management. I'm afraid size all but kills this option.


                        Which would you choose?

                        • 9. Re: Re: audio control and sync on mobile
                          ChipNow Level 1

                          Much of this comes from the <audio>.autoplay ban on all iOS browsers and some (not all) Android browsers. I have written some test code to check for the capability. I have submitted it to Modernizr and to stack exchange. I have also put in a request to the Animate team to either ask for a click in the preloader (so the audio can load) or let me get it and then deal with the necessary src switching so that Animate will run on mobile as it does on the desktop. This is doable - particularly since we now know that mobile can support multiple <audio> playing at once - though multi-layered audio on mobile is another topic. Better than nothing. Just like everything else, getting it to work is, well, work.


                          Thanks again to jbowden for hanging in there.