6 Replies Latest reply on Nov 23, 2009 11:36 AM by bringrags

    Video Capture Image

    tyjohnston Level 1

      I have 3 scenarios where I need to display an image at the start of a video, before play() is called.

       

      - Remote Image and HTTP Video Stream

      - RTMP Frame time ( frame of RTMP stream to show before video is played ) and RTMP Stream

      - Remote Image and HTTP Audio Stream

       

      Is this where a SerialElement would benefit?  I need to have the image or RTMP frame shown before the user hits play().

       

      Also, I usually call play( streamName, captureTime, 0 ) if I want to show a frame of the RTMP stream.  How is this functionality served in OSMF?

       

      Thanks in advance!

        • 1. Re: Video Capture Image
          bringrags Level 4

          The answer might be different for each of these, let me take a crack:

           

          - Remote Image + HTTP Video Stream:  This is an ideal candidate for a SerialElement, where the first child is an ImageElement and the second a VideoElement.  The problem (as I just discovered from trying this) is that the first child isn't playable, so you won't be able to play() it.  The solution for this is to create an ImageElement subclass which adds the IPlayable trait.  I've added an example to the ExamplePlayer (search for "Poster Frame") to demonstrate this.  I'm pretty sure there are some bugs to work through here, but hopefully this is a good start.

          - RTMP Frame + RTMP Stream:  This one is a bit trickier.  Again, a SerialElement is most likely the solution, this time with the first child being a VideoElement that includes subclip metadata of zero duration (see the very-brand-new Subclip Spec for details on how to define subclip metadata).  The problem is that unlike ImageElement, VideoElement doesn't automatically load and display the first frame, so the first (subclip) child won't display as a poster frame.  I'm not quite sure what the best way to address this is.  It might be possible to create a subclass of VideoElement (RTMPPosterFrameElement?) that does a play/pause immediately after being loaded.  I'll add this to my list of good use cases to attempt (but feel free to try yourself and post the results).

          - Remote Image + HTTP Audio Stream:  This might be the same as the first case (if the goal is to show a poster frame prior to playback).  But if the goal is to show the image the entire time that the audio stream plays, then you should use a ParallelElement to wrap an ImageElement and an AudioElement (the latter with a SoundLoader).

           

          As for your last question, play(streamName, captureTime, 0) maps to the following (again, using the new Subclip API):

           

          var resource:URLResource = new URLResource(new FMSURL(streamName));
          var kvFacet:KeyValueFacet = new KeyValueFacet(MetadataNamespaces.SUBCLIP_METADATA);
          kvFacet.addValue(MetadataNamespaces.SUBCLIP_START_ID, captureTime);
          kvFacet.addValue(MetadataNamespaces.SUBCLIP_END_ID, captureTime);
          resource.metadata.addFacet(kvFacet);
          var videoElement:VideoElement = new VideoElement(new NetLoader(), resource);

          1 person found this helpful
          • 2. Re: Video Capture Image
            tyjohnston Level 1

            For the RTMPPosterFrameElement, which methods am I able to override?  The VideoElement does not really expose any of it's functions.  Maybe override processReadyState()?  Maybe the VideoElement needs to expose the NetStream somehow?

            • 3. Re: Video Capture Image
              tyjohnston Level 1

              Ok, so what I've done, is subclassed LoadableMediaElement and used most of the VideoElement functions, and made a few modifications:

               

              The constructor accepts a start time and defines the subclip metadata of the resource:

               

              public function RTMPPosterFrameElement(loader:NetLoader, start:int, resource:IMediaResource=null)
              {
                   super(loader, resource);
                             
                   var kvFacet:KeyValueFacet = new KeyValueFacet(MetadataNamespaces.SUBCLIP_METADATA);
                   kvFacet.addValue(MetadataNamespaces.SUBCLIP_START_ID, start);
                   kvFacet.addValue(MetadataNamespaces.SUBCLIP_END_ID, start);
                   resource.metadata.addFacet(kvFacet);
              }
              

               

              Then, I called play() at the bottom of finishLoading():

               

              (getTrait(MediaTraitType.PLAYABLE) as IPlayable).play();
              

               

               

              I also have a trait which subsclasses NetStreamPlayableTrait and exposes a few of its variables and the onNetStatus function:

               

               

              public class RTMPPosterFramePlayableTrait extends NetStreamPlayableTrait
                   {
                        public function RTMPPosterFramePlayableTrait(owner:MediaElement, netStream:NetStream, resource:IMediaResource)
                        {
                             super(owner, netStream, resource);
                        }
                        
                        override protected function onNetStatus(event:NetStatusEvent):void
                        {
                             switch (event.info.code)
                             {
                                  case NetStreamCodes.NETSTREAM_PLAY_FAILED:
                                  case NetStreamCodes.NETSTREAM_PLAY_FILESTRUCTUREINVALID:
                                  case NetStreamCodes.NETSTREAM_PLAY_STREAMNOTFOUND:
                                  case NetStreamCodes.NETSTREAM_PLAY_NOSUPPORTEDTRACKFOUND:                    
                                  case NetStreamCodes.NETSTREAM_FAILED:
                                       netStream.pause();
                                       streamStarted = false;
                                       resetPlaying();                         
                                       break;
                                  case NetStreamCodes.NETSTREAM_PLAY_STOP:
                                       var pausable:IPausable = owner.getTrait(MediaTraitType.PAUSABLE) as IPausable;
                                       if (pausable != null)
                                       {
                                            pausable.pause();
                                       }          
                                       break;
                             }
                        }
                        
                   }
              

               

              This would be an easier implementation if NetstreamPlayableTrait initially exposed onNetStatus(), var owner, var streamStarted, var netstream, and if VideoElement exposed finishLoading().

               

              The AllExamples.as list example looks like this:

               

              examples.push
                   ( new Example
                   (      "RTMP Poster Frame"
                   ,      "Demonstrates the use of a SerialElement to present an RTMP poster frame prior to playback."
                   ,       function():MediaElement
                        {
                             var serialElement:SerialElement = new SerialElement();
                             serialElement.addChild(new RTMPPosterFrameElement(new NetLoader(), 30, new URLResource(new FMSURL(REMOTE_STREAM))));
                             serialElement.addChild(new VideoElement(new NetLoader(), new URLResource(new FMSURL(REMOTE_STREAM))));
                             return serialElement; 
                        } 
                   )
                   );
              

               

              What do you think?

              • 4. Re: Video Capture Image
                bringrags Level 4

                I think there's a simpler way, based on removing some traits in a VideoElement subclass.  My approach is similar, in that I subclass VideoElement and override processReadyState, but the processReadyState method doesn't have to access the details of NetStream:

                 

                        override protected function processReadyState():void
                        {
                            super.processReadyState();

                 

                            // First, remove the temporal and playable traits.  Doing so
                            // will ensure that our "play" call to display the poster
                            // frame won't cause this MediaElement to complete (and therefore
                            // trigger the playback of the next child, when in a SerialElement).
                            removeTrait(MediaTraitType.TEMPORAL);
                            var playable:IPlayable = removeTrait(MediaTraitType.PLAYABLE) as IPlayable;
                           
                            // Calling play() on our removed trait will cause the poster frame
                            // to be displayed.  But because this playable trait is detached, no
                            // events are dispatched to the client.  From a traits perspective,
                            // this is functionally equivalent to an ImageElement, where there's
                            // an IViewable but no IPlayable.
                            playable.play();
                           

                            // Last, to ensure that the user can complete playback of this item,
                            // we add a dummy IPlayable trait.
                            addTrait(MediaTraitType.PLAYABLE, new PosterFramePlayableTrait(this));
                        }

                Code is checked in here, and ExamplePlayer now has an "RTMP Poster Frame" example.

                • 5. Re: Video Capture Image
                  tyjohnston Level 1

                  Brian, thank you for your help.  I am getting more and more comfortable with the syntax and flow of the framework.  Well done!

                   

                  Is it possible that this framework (or the foundation of) will be cached by the Flash Player at some point (like the Flex Framework)?

                  • 6. Re: Video Capture Image
                    bringrags Level 4

                    Yes, we're investigating getting OSMF cached by the player after we release.