10 Replies Latest reply on Jul 9, 2012 7:21 AM by Tim Beynart

    cuepoints for dyn http stream (f4m)




      I am trying to add(server side)/extract(clientside) Cuepoints or other Metadata related to a dynamic http streamed live video with variable bitrate (and also single stream). Any idea how to do this?



        • 1. Re: cuepoints for dyn http stream (f4m)
          weizhangstrobe Level 2

          I got some information from my colleague. See if it answers your question.


          The underlying logic for AS cue points in OSMF can be found in TimelineMetadata in the org.osmf.metadata package.  If you take a look at the addMarker method, you'll see this ASDoc comment:



                   * Adds the specified TimelineMarker to this object.  This class

                   * maintains the TimelineMarkers in time order.  If another TimelineMarker

                   * with the same time value exists within this object, then the existing

                   * value will be overwritten.


                   * @param marker The marker to add.


                   * @throws ArgumentError If marker is null or specifies an invalid time.



          If you need more data for a single cue point, you can add that to the Object parameter when you create the CuePoint.  Notice the 4th param in the CuePoint constructor is an Object. This can be anything you want, such as:


          var cuePoint:CuePoint = new CuePoint(CuePointType.ACTIONSCRIPT, 162.12, "MyCuePoint", {key1:"value 1", key2:"value 2"});


          -Wei Zhang

          Senior Computer Scientist   

          Adobe Systems, Inc.


          • 2. Re: cuepoints for dyn http stream (f4m)
            OrcoTMM55 Level 1

            Hello Wei


            Thanks for the response.

            I have found the TimelineMarker and Cuepoint packages. However, it seemed they only allow you to create temporary Metadata* which is restricted to the client running OSMF. What I was looking for was something that I can add on the serverside and push through to the clients. Appreciate your help.


            In case it is working like the .send() of the netstream class, where would I add the listener?


            * I am getting confused with the use of the term Metadata as it seems to mean several things (osmf meta= widget or elemt parameters, metadata for dyn streaming of adobe is the stream information and so on)





            • 3. Re: cuepoints for dyn http stream (f4m)
              OrcoTMM55 Level 1

              ok, another day gone and no further.


              I have found the onMetaData and onCuePoint methods in the LightweightVideoElement. However, the documentation says this only works with progressive and simple rtmp streaming (that would indicate no http). Please do tell me if I am confusing things here.



              • 4. Re: cuepoints for dyn http stream (f4m)
                rshin Level 3

                Hi Oscar,


                Now we see what your real question was.  That would be more so question to FMS server side (http streaming packager).

                There is a separate form called "http dynamic streaming" mainly designed for http streaming (server, client).


                I will also ping FMS server folks to see if they have further guidedance...



                • 5. Re: cuepoints for dyn http stream (f4m)
                  rshin Level 3

                  FMS team confirmed that http streaming cuepoint works just like rtmp streaming on server side as well.


                  For VOD case, you'd use a tool that can inject cuepoints into your content (flv, f4v) before they are packaged into http streaming content.

                  If the source file has cuepoints, they will automatically get sent with A/V content.


                  For Live case, you could use server side action script to do a Stream.send() into the live stream. The data message will get recorded into F4F(http streaming) along with the rest of the content.   If a data message called "foo" is sent, you'd create a handler on client "foo" that would get dispatched when the data message arrives.  Please refer to the AS docs for NetStream.


                  The following client-side code broadcasts the message "Hello world" to the foo handler function on each client that is connected to myApp:

                  nc = new NetConnection(); 
                  ns = new NetStream(nc);
                  ns.send("foo", "Hello world");

                  • 6. Re: cuepoints for dyn http stream (f4m)
                    OrcoTMM55 Level 1

                    Hello Ryan


                    Thanks for your reply. This framework is fairly new to me and I am still in the process of understanding everything so I understand that my question was a little unclear.

                    I have tried to dosomething like you below in a custom webcam recorder, stream it to the server and then stream it via dyn http to a client but wasn't able to call the custom function.

                    If you could point me to the location in the osmf files where I had to add that handler for your particular example, I'd be very greateful.


                    Many thanks.


                    Kind Regards,

                    Oscar Kazmeier


                    Oscar Kazmeier

                    Flex/Flash Developer


                    Stream UK


                    • 7. Re: cuepoints for dyn http stream (f4m)
                      rshin Level 3

                      For VOD:

                      Server side: you'd have to inject your cuePoint into your content (FLV, F4V) then streaming just like any other contents from FMS.

                      Client Side:  If you are using OSMF player, please refer to the above post (Wei's), and I'd look at CuePoint Sample in SVN.




                      From SVN depot,  osmf\public\trunk\apps\samples\framework\CuePointSample


                      Look at onMetadataAdd, onCuePoint as to how OSMF handles differen types cuePoints.


                      For Live content,


                      Server side needs a server side script handling to inject your cuepoint into live content.  I've examplified a sample you can start off. (See below)

                      Basically you'd use netStream.send to your callback function to inject a cuePoint message.

                      Note though if you are using OSMF based player to handle this in live content, you may want to send the cuePoint data as an object that complies with 'CuePoint" class (base class for CuePoint feature in OSMF).  If you are using your own player, you could send any type of data you'd fit into your player.


                      // main.asc file
                      // server-side actionscript


                      var nc;
                      var ns;


                      application.onAppStart = function()




                      // Called when a client connects
                      application.onConnect = function(client) {
                           // accept the new client's connection
                            // send a message
                            trace(client.id + " is connected");   


                      // Called when a client disconnects
                      application.onDisconnect = function(client) {
                           trace(client.id + "disconnecting at " + new Date());


                      function checkForEvent()
                           info = new Object();

                      //Just to construct CuePoint Object.  Use the appropriate object construction as you need.
                           info.type = "LiveCutPoint";
                           info.time = 2;
                           info.name = "CuePointString";
                           info.parameters = null;
                           ns.send("onCuePoint", info);
                           trace("cuePoint Sent");




                      // Called when the client publishes
                      application.onPublish = function(client, myStream) {


                          trace(myStream.name + " is publishing into application " + application.name);
                           ns = myStream;
                           application.timer_int = setInterval(checkForEvent, 5000);




                      application.onUnpublish = function( client, myStream ) {


                          trace(myStream.name + " is unpublishing");




                          trace("clear Interval timer int");




                      On Client side, OSMF has implemented onCuePoint funtion in LightWeightVideoElement already, you could just use "onCuePoint" when you send cuePoint data using the callback function. (see above server side script, stream.send()).

                      Please refer to Ln 588 in LightWeightVideoElement for details if you are using your own player.

                      CuePointSample should work with live content as well with minor modification.


                      Instead of //var mediaElement:MediaElement = new VideoElement(new URLResource(REMOTE_STREAM_WITH_NAV));


                      Use mediaFactory as followings in Init() in CuePointSample.mxml in order to play Live Content.


                                    var factory:MediaFactory = new DefaultMediaFactory();
                                      var url:StreamingURLResource = new StreamingURLResource(REMOTE_STREAM_WITH_NAV, StreamType.LIVE_OR_RECORDED);
                                      var mediaElement:MediaElement = factory.createMediaElement(url);


                      This should get you at least to print out cuePoint once for live content in the table. You may want to further modification to see all timeline metadata in UI.


                      For http streaming, it should work just like rtmp does.





                      • 8. Re: cuepoints for dyn http stream (f4m)
                        daslicht Level 2

                        >you'd use a tool

                        Which tool?


                        How to add cuepoints to prerecorded material?


                        Would I have to add them to a flv and than convert it somehow ?



                        How about the cuePoint Player example, it only shows how to play a rtmp stream not a f4m dynamic http stream.


                        Any Example please !



                        • 9. Re: cuepoints for dyn http stream (f4m)



                          I know this thread is quite old, but I'm looking for exactly this feature and I haven't found anything more current.


                          Actually, I use NetStream.send to attach cuepoints for RTMP live streamings. But now, I want use this tehcnique for HTTP Dynamic streaming, specifically, for make a slides sync player for HTML5 iOS compatible.


                          FMS staff confirm that is possible to use NetStream.send to inject cuepoints to HTTP Dynamic streaming, But, here is my question, Is it possible to extract the messages with HTML5?


                          Thank you,



                          Carles Galan Cladera.


                          • 10. Re: cuepoints for dyn http stream (f4m)
                            Tim Beynart Level 1

                            Carles, I am researching the same subject. As far as I can discern from hours of searching, there is no way to read id3 tags from an HTTP stream in the browser. You will have to use native code in iOS. If I am wrong, someone please reply with a correction.

                            The PDF for iOS docs is here:

                            http://developer.apple.com/library/ios/documentation/AudioVideo/Conceptual/HTTP_Live_Strea ming_Metadata_Spec/HTTP_Live_Streaming_Metadata_Spec.pdf

                            On eexample of the bad news: