6 Replies Latest reply on Jun 7, 2010 8:51 AM by bringrags

    Building a SMIL like Plugin

    shrelp

      I need to develop a plugin that is very similular to the SMIL plugin.

      Composition Plugin (SMIL) and Use of [LoadableProxyElement]

      A SMIL plugin needs to parse a SMIL document, and create a composite media element. However, a SMIL plugin does not knowapriori which [MediaElement] needs to be created before parsing through the SMIL document. As a result, it does not know which[MediaElement] to include with its [MediaInfo] that gets registered with the Strobe [MediaFactory]. To solve this problem, we recommend using the [ProxyElement] class as its default [MediaElement]. Here's how the approach might look like

      1. SMIL plugin's [PluginInfo] class (to be precise, its mediaInfoSet getter method) will return an array containing a single[MediaInfo] object. The [MediaInfo] object will contain '[LoadableProxyElement]' as its [MediaElement], 'URLResource' as the [IMediaResource] and 'SMILLoader', which extends [LoaderBase].
      2. The SMILLoader class will have a 'load' method. Here's what it should do at a high level
        1. Download the SMIL file specified in the URLResource object
        2. Parse the SMIL document
        3. Create a [SerialElement], or a [ParallelElement] or a specific [MediaElement] (such as [VideoElement]) depending what's at the root level in the SMIL document
        4. Call 'load' on its ILoadable trait
        5. Wait for it to be loaded
        6. Use the [ILoadedContext] object from the loaded media element and include it in the LOADED event.

      http://opensource.adobe.com/wiki/display/osmf/Plugin+Support+and+API+Specification

       

       

      My plugin needs to take an asset id, make a request a to get the FLV url, then create a Proxy element to monitor the playback.

       

      I'm like the player implementation would look like this.

       

      // VeohResource would append the asset id to the full service url.
      var resource:IURLResource     = new URLResource(new VeohResource("v123456"));
      var mediaElement:MediaElement = mediaFactory.createMediaElement(resource);
      mediaPlayerWrapper.element    = mediaElement;
      

       

      The closest plugin example I could find was the SMIL plugin but after many many hours of reading contradictory blog post and different sprint code I'm going to need some help.

       

      1. Take down the old documentation to pervious sprints.

      2. Where is the SMIL sample project?  All of the other plugins have a sample project.  The unit test project was helpful but didnt' provide any clues on player implementation.

       

      So from the looks of this document I guess you just have to load the plugin and place this code in the onPluginLoaded handler?

       

           var resource:IURLResource     = new URLResource(new URL("http://myserver/myfile.smil"));
           var mediaElement:MediaElement = mediaFactory.createMediaElement(resource);
           mediaPlayerWrapper.element    = mediaElement;
      

       

       

       

       

      SMIL Plugin Questions

      1.  I was going to ask how to call SMILLoader.load method from the player code,

        http://opensource.adobe.com/svn/opensource/osmf/tags/sprint9-stable/plugins/SMILPlugin/org /osmf/smil/loader/SMILLoader.as

      but now in the latest release it has been renamed to executeLoad, which leads me to believe this method will be executed

      by the framework and not the player code.

      http://opensource.adobe.com/svn/opensource/osmf/trunk/plugins/SMILPlugin/org/osmf/smil/loa der/SMILLoader.as

       

      So what exactly is calling SMILLoader.executeLoad? and what player code triggers this call?

       

       

      2.  Why does SMILLoader get created twice in SMILPluginInfo.

      Arr, looks like this may only be an issue with the sprint 9 tag.

      http://opensource.adobe.com/svn/opensource/osmf/tags/sprint9-stable/plugins/SMILPlugin/org /osmf/smil/SMILPluginInfo.as

      http://opensource.adobe.com/svn/opensource/osmf/trunk/plugins/SMILPlugin/org/osmf/smil/SMI LPluginInfo.as

       

       

      3.  I'm assuming after loading the SMILPlugin calling mediaFactory.createMediaElement(new URLResource(new URL("http://myserver/myfile.smil"))) would some how magically invoke SMILLoader.executeLoad?

       

       

       

      4. In SMILLoader.finishLoad

      a) Where did this loadTrit come from? 

      var loadedElement:MediaElement = mediaGenerator.createMediaElement(loadTrait.resource, smilDocument, factory);

      And is the resource a reference to the original resource from the player code?

      this one => var resource:IURLResource = new URLResource(new URL("http://myserver/myfile.smil"));

       

       

      b) So I'm trying to figure out happens after the loadedElement var is assigned a new MediaElement.

      This LoadedFromDocumentLoadTrait method is new to sprint 10 and I have no idea what it does?

       

       

      c) So I'm completely confused here. 

       

      It looks like the MediaElement is created and it's now time to get it

         back to the player code so it could be added to a MediaPlayer, right?

       

      So in sprint-9 I thought the player code had to listen for this "update load trait" event to get the MediaElement

      but now in sprint-10 there is a whole object and event handler.

       

      So how the hell does the player get the MediaElement created here?

       

       

      5. The example player code provided doesn't jive with the implementation found in the trunk.

      http://opensource.adobe.com/wiki/display/osmf/SMIL+Support

       

       

      // If we define our resource here

      var resource:IURLResource = new URLResource(new URL("http://myserver/myfile.smil"));

      // How could have the SMIL document been loaded and parsed before we call this line?

      var mediaElement:MediaElement = mediaFactory.createMediaElement(resource);

      mediaPlayerWrapper.element = mediaElement;

       

       

      It seems the player code needs to listen for a SMIL loaded event before doing anything with

      the mediaElement, right?

       

       

       

       

       

      Thanks in advance for any advice you can give me.

        • 1. Re: Building a SMIL like Plugin
          bringrags Level 4

          Wow, a lot to digest there.  I''ll give a few high-level answers first.

           

          When in doubt, follow the APIs, code, and sample apps, rather than what's in the specs.  Many of the specs were written several sprints ago, when names were different.  We're planning to update before 1.0, but until then they're best used as high-level references.

           

          The AkamaiPluginSample shows how to load and use a number of plugins, including the SMIL plugin.  I'd definitely recommend stepping through the loading and playback of a SMIL document.

           

          In terms of the model, think of a LoadFromDocumentElement as a placeholder for an undetermined MediaElement.  You can work with a LoadFromDocumentElement as if it's a regular MediaElement, but once the document gets loaded, the element takes on the properties of the root MediaElement generated from the document.  It's like the old switcheroo.

           

          To your questions:

           

          1. executeLoad is the hook where subclasses of a LoaderBase implement their loading logic.  Yes, the framework calls it.  If you're implementing your own loader, you should override it and do your custom logic there.

          3. The createMediaElement will return the MediaElement.  Later on, when it's loaded (which happens implicitly when you pass it to a MediaPlayer), executeLoad will get called.

          4a. Yes, this is the original resource (the SMIL URL).

          4b. The loadedElement variables holds the root MediaElement that is generated from a SMIL document.  For example, if the SMIL document has a <seq> tag with three <video> children, the root element will be a SerialElement with three VideoElement children.  This is the element that will end up as the proxiedElement of the SMILElement.

          4c. The loadedElement is assigned to the LoadFromDocumentLoadTrait, which is the LoadTrait for the SMILElement.  If you're creating your own SMIL-like plugin, then your responsibility is simply to assign the generated MediaElement to this trait, and then signal that the object is ready (by calling updateLoadTrait with state READY).

          5. The player code should be able to work with a SMILElement as if it's any other MediaElement.  In other words, they load it, call play, etc.  All of the complexity should be isolated to the SMIL plugin.

          1 person found this helpful
          • 2. Re: Building a SMIL like Plugin
            shrelp Level 1

            Thanks for your response.  I thought that AkamaiPluginSample project was specific to the Akamai CDN. 

             

            I'm trying to create a minimalist SMIL example and I can't figure out what event tells me mediaPlayer.displayObject is not NULL.

             

             

            package
            {
                 import flash.display.Sprite;
                 
                 import org.osmf.events.MediaFactoryEvent;
                 import org.osmf.media.MediaElement;
                 import org.osmf.media.MediaFactory;
                 import org.osmf.media.MediaPlayer;
                 import org.osmf.media.PluginInfoResource;
                 import org.osmf.media.URLResource;
                 import org.osmf.net.StreamingURLResource;
                 import org.osmf.smil.SMILPluginInfo;
                 import org.osmf.smil.elements.SMILElement;
            
                 public class HelloWorld2 extends Sprite
                 {
                      
                      public function HelloWorld2()
                      {
                           mediaPlayer      = new MediaPlayer();
                           mediaFactory      = new MediaFactory();
                           mediaFactory.addEventListener( MediaFactoryEvent.PLUGIN_LOAD, handle_pluginLoad);
                           mediaFactory.loadPlugin( new PluginInfoResource( new SMILPluginInfo ) );
                      }
                      private function handle_pluginLoad(e:MediaFactoryEvent):void
                      {
                           trace("handle_pluginLoad");
                           var resource:URLResource = new StreamingURLResource(SMIL);
                           mediaFactory.addEventListener( MediaFactoryEvent.MEDIA_ELEMENT_CREATE, handle_mediaElementLoaded);
                           mediaElement = mediaFactory.createMediaElement(resource);     
                      }
                      private function handle_mediaElementLoaded(e:MediaFactoryEvent):void
                      {
                           var smilElement:SMILElement = e.mediaElement as SMILElement;
                           trace("handle_mediaElementLoaded");
                           mediaPlayer.autoPlay = true;
                           mediaPlayer.media = e.mediaElement;
            // HOW DO I LISTEN FOR A DISPLAY
            // OBJECT BEING SET ON THE MEDIAPLAYER? 
                           addChild(mediaPlayer.displayObject);
                      }
                      private var mediaPlayer:MediaPlayer;
                      private var mediaFactory:MediaFactory;
                      private var mediaElement:MediaElement;
                      public static const SMIL:String = "http://mediapm.edgesuite.net/osmf/content/test/smil/elephants_dream.smil";
                 }
            } 
            
            • 3. Re: Building a SMIL like Plugin
              shrelp Level 1

              Okay I'm stumped.  How do I get a SMILElement that looks to have loaded correctly added to a MediaPlayerso I can add the mediaPlayer.displayObject to the stage?

               

              I've gone through the AkamaiPluginSample forward and backwards with no luck.  Thanks for your help.

               

               

              package
              {
                   import flash.display.Sprite;
                   import flash.events.Event;
                   
                   import org.osmf.events.ContainerChangeEvent;
                   import org.osmf.events.MediaElementEvent;
                   import org.osmf.events.MediaErrorEvent;
                   import org.osmf.events.MediaFactoryEvent;
                   import org.osmf.events.MediaPlayerStateChangeEvent;
                   import org.osmf.events.MetadataEvent;
                   import org.osmf.media.MediaElement;
                   import org.osmf.media.MediaFactory;
                   import org.osmf.media.MediaPlayer;
                   import org.osmf.media.PluginInfoResource;
                   import org.osmf.media.URLResource;
                   import org.osmf.net.StreamingURLResource;
                   import org.osmf.smil.SMILPluginInfo;
                   import org.osmf.smil.elements.SMILElement;
              
                   public class HelloWorld2 extends Sprite
                   {
                        
                        public function HelloWorld2()
                        {
                             mediaPlayer      = new MediaPlayer();
                             mediaPlayer.addEventListener(MediaPlayerStateChangeEvent.MEDIA_PLAYER_STATE_CHANGE,
                             function(e:Event):void {
                                  trace(e.type, mediaPlayer.displayObject);
                             });
                             mediaFactory      = new MediaFactory();
                             mediaFactory.addEventListener( MediaFactoryEvent.PLUGIN_LOAD, handle_pluginLoad);
                             mediaFactory.loadPlugin( new PluginInfoResource( new SMILPluginInfo ) );
                        }
                        private function handle_pluginLoad(e:MediaFactoryEvent):void
                        {
                             function handler(e:Event):void { trace("_",e.type, "mp",mediaPlayer.displayObject); }
                             trace("handle_pluginLoad");
                                            
                             mediaFactory.addEventListener( MediaFactoryEvent.MEDIA_ELEMENT_CREATE, handle_mediaElementLoaded);
                             var resource:URLResource = new StreamingURLResource(SMIL);
                             mediaElement = mediaFactory.createMediaElement(resource);
              //               mediaElement = mediaFactory.createMediaElement(resource);
              //               mediaElement.addEventListener(MediaErrorEvent.MEDIA_ERROR, handler);     
              //               mediaElement.addEventListener(ContainerChangeEvent.CONTAINER_CHANGE, handler);     
              //               mediaElement.addEventListener(MediaElementEvent.METADATA_REMOVE, handler);     
              //               mediaElement.addEventListener(MediaElementEvent.TRAIT_ADD, handler);     
              //               mediaElement.addEventListener(MediaElementEvent.TRAIT_REMOVE, handler);
                        }
                        private function handle_mediaElementLoaded(e:MediaFactoryEvent):void
                        {
                             var smilElement:SMILElement = e.mediaElement as SMILElement;
                             trace(smilElement.proxiedElement);
                             mediaFactory.addEventListener(MediaFactoryEvent.MEDIA_ELEMENT_CREATE, function(e:Event):void {
                                       trace("MEDIA_ELEMENT_CREATE **** LOADED!!!!!", mediaPlayer.displayObject);
                             });
                             
                             smilElement.addEventListener(ContainerChangeEvent.CONTAINER_CHANGE, function(e:Event):void { trace("", e.type); });
                             smilElement.addEventListener(Event.ACTIVATE, function(e:Event):void { 
                                  trace("", e.type); 
                             });
                             smilElement.addEventListener(Event.DEACTIVATE, function(e:Event):void { trace("", e.type); });
                             smilElement.addEventListener(MediaElementEvent.METADATA_ADD, function(e:Event):void { trace("", e.type); });
                             smilElement.addEventListener(MediaElementEvent.TRAIT_ADD, function(e:Event):void { trace("", e.type); });
                             smilElement.addEventListener(MediaErrorEvent.MEDIA_ERROR, function(e:Event):void { trace("", e.type); });
                             smilElement.metadata.addEventListener(MetadataEvent.VALUE_ADD, function(e:Event):void {
                                  trace("VAL ADDED",mediaPlayer.media);
                             }     );
                             /* smilElement.addEventListener(MediaElementEvent.METADATA_ADD, function(e:Event):void {
                                  trace("TRAIT_ADD", mediaPlayer.displayObject);
                             });
                             smilElement.addEventListener(ContainerChangeEvent.CONTAINER_CHANGE, function(e:Event):void {
                                  trace("CONTAINER_CHANGE", mediaPlayer.displayObject);
                             });
                             trace("handle_mediaElementLoaded", e.mediaElement.resource); */
                             
                             mediaPlayer.autoPlay = true;
                             /* mediaPlayer.addEventListener(MediaElementChangeEvent.MEDIA_ELEMENT_CHANGE,
                             function(e:Event):void {
                                  trace("");
                                       trace("LOADED!!!!!", mediaPlayer.displayObject);
                             }); */
              //               mediaPlayer.media = smilElement;
              //               mediaPlayer.media = new VideoElement(new URLResource("http://mediapm.edgesuite.net/strobe/content/test/AFaerysTale_sylviaApostol_640_500_short.flv"));
                             
                             /* stage.addEventListener(Event.ENTER_FRAME, function(e:Event):void {
                                  if(mediaPlayer.displayObject) {
                                       trace("**** LOADED!!!!!", mediaPlayer.displayObject);
                                  }
                             }); */
                             
              //               addChild(mediaPlayer.displayObject);
                        }
                        private var mediaPlayer:MediaPlayer;
                        private var mediaFactory:MediaFactory;
                        private var mediaElement:MediaElement;
                        public static const SMIL:String = "http://mediapm.edgesuite.net/osmf/content/test/smil/elephants_dream.smil";
                   }
              }
              

              • 4. Re: Building a SMIL like Plugin
                bringrags Level 4

                You should be able to listen to the MediaPlayerStateChangeEvent, and add MediaPlayer.displayObject to the stage when you get a "READY" state change.  (Alternatively, you could use MediaPlayerSprite, which manages the addition of the display object itself.)  Here's an example based on your code:

                 

                    public class HelloWorld2 extends Sprite
                    {
                          public function HelloWorld2()
                          {
                               mediaPlayer      = new MediaPlayer();
                               mediaPlayer.autoPlay = true;
                               mediaPlayer.addEventListener(MediaPlayerStateChangeEvent.MEDIA_PLAYER_STATE_CHANGE, handle_stateChange);
                               mediaFactory      = new DefaultMediaFactory();
                               mediaFactory.addEventListener( MediaFactoryEvent.PLUGIN_LOAD, handle_pluginLoad);
                               mediaFactory.loadPlugin( new PluginInfoResource( new SMILPluginInfo ) );
                          }
                          private function handle_pluginLoad(e:MediaFactoryEvent):void
                          {
                               trace("handle_pluginLoad");
                               var resource:URLResource = new StreamingURLResource(SMIL);
                               mediaElement = mediaFactory.createMediaElement(resource);
                               mediaPlayer.media = mediaElement;
                          }
                          private function handle_stateChange(e:MediaPlayerStateChangeEvent):void
                          {
                                 if (e.state == MediaPlayerState.READY)
                                 {
                                   trace("handle_stateChange");
                                   addChild(mediaPlayer.displayObject);
                               }
                          }
                          private var mediaPlayer:MediaPlayer;
                          private var mediaFactory:MediaFactory;
                          private var mediaElement:MediaElement;
                          public static const SMIL:String = "http://mediapm.edgesuite.net/osmf/content/test/smil/elephants_dream.smil";
                    }

                • 5. Re: Building a SMIL like Plugin
                  ionflow

                  Can you explain why in this example you are setting the player visibility using:

                   

                   

                   private function onMediaPlayerStateChange(event:MediaPlayerStateChangeEvent):void
                   {
                       //media player state change == READY not always being called after ImageElement is created.
                       //but it seems to always dispatch the PLAYING event
                        trace("player state changed to:"+event.state);
                        if(event.state == MediaPlayerState.READY)
                        {
                            addChild(player.displayObject);
                        }
                   }
                  

                   

                   

                   

                  Instead of setting the media property after adding the SMIL element to the container like this:

                   

                   

                  private function handle_pluginLoad(e:MediaFactoryEvent):void
                   {
                       var resource:URLResource = new StreamingURLResource(SMIL);
                        element = factory.createMediaElement(resource);
                        element.addEventListener(MediaErrorEvent.MEDIA_ERROR, onMediaError, false, 0, true);
                   
                        container.addMediaElement(element);
                   
                        player.media = element;
                   }
                  

                   

                   

                  With your example, image files are not always shown because they do not seem to dependably dispatch READY events after loading.

                   

                  Thanks!

                  • 6. Re: Building a SMIL like Plugin
                    bringrags Level 4

                    The two approaches should be functionally equivalent, although I think your approach looks a bit cleaner in that the event management is internalized to the container class.  If there are cases where you don't get a READY event on loading an image (or any other media type), then that would be a bug (and please file!).