13 Replies Latest reply on Nov 1, 2012 5:07 PM by Michael Thornburgh

    NetGroup multicast: delay, a/v out of sync

    aylarov Level 1

      Hello,

       

      I've tested multicast example Michael provided us with (viewer.mxml/provider.mxml) and found that after viewer is connected there is a significant delay between publisher and viewer (around few secs) that is being reduced till realtime in some time, but even after delay is reduced we have audio/video out of sync, sound came later than video (or maybe video came later than audio... don't remember exactly).

       

      Do you have any ideas why this happens - maybe because of beta?

        • 1. Re: NetGroup multicast: delay, a/v out of sync
          Michael Thornburgh Adobe Employee

          a delay of up to several seconds (up to NetStream.multicastWindowDuration + NetStream.bufferTime, but hopefully much less) is to be expected, between the publisher when-it-happens and subscriber when-you-see-it.

           

          A/V should be in sync, though.  we have seen audio and video out of sync when Flash Player is running in a VM (like windows on vmware).  is A/V always out of sync for you, or just sometimes?  is it *really* out of sync (like seconds) or is it a little out of sync (up to a second)?  are you on mac, windows, linux?  Flash Player or AIR?  browser or standalone?  are you running in a virtualized OS?  and are you using the latest Flash Player beta (beta 2)?  finally, when you see audio and video out of sync, check your CPU usage.  if your CPU is maxed out, then it'll be hard to maintain sync (and maintain framerate).

           

          -mike

          1 person found this helpful
          • 2. Re: NetGroup multicast: delay, a/v out of sync
            aylarov Level 1

            Michael,

             

            Thanks for tip about mutlicastWindowDuration, it helped to reduce delay

             

            About out of sync problem - I was streaming from MacOSX my video to 2 PC - 1 with Vista, 2nd with Windows7, no virtual machines involved. All computers had beta2 installed (Flash Player app, not AIR), CPUs were ok, because laptops are powerful ones. Audio/Video were our of sync sometimes, I guess not always, it was around a sec of difference. I will try to repeat this tests and provide you with more results.

            • 3. Re: NetGroup multicast: delay, a/v out of sync
              Michael Thornburgh Adobe Employee

              just to clarify: by "out of sync" do you mean "on one playback computer, audio and video were out of sync", or do you mean "one playback computer was a second ahead (or behind) another playback computer"?  you should not expect the playback computers to be in sync with each other (again, maximum jitter in the system is related to those two properties i listed earlier).  since you have two playback computers going, please make sure you're not hearing the audio from one and thinking it's supposed to go with the video from the other. 

               

              oh also, NetStream.bufferTime should not be 0 for multicast.  the multicast system can have a lot of jitter (Matthew's 2009 MAX presentation's explanation of our P2P multicast system should illustrate why there can be jitter), so the NetStream should be in "buffered" mode for best results.

               

              finally: i recommend not changing the NetStream.multicast* properties.  you can, of course, but they have subtle interactions and we have not yet produced a companion document that really goes into detail about what they all are, what they do, and how they affect the multicast mesh.  for example: if you reduce multicastWindowDuration to below the multicastFetchPeriod (or the multicastAvailabilityUpdatePeriod) then the "pull" mode of multicast won't work effectively.  changing these properties may dramatically increase the amount of overhead traffic generated and may result in a considerable amount of duplicated media packets, which will also increase the bandwidth used for the multicast.

               

              feel free to experiment, of course, but i highly recommend staying with the defaults at this time.

               

              -mike

              1 person found this helpful
              • 4. Re: NetGroup multicast: delay, a/v out of sync
                aylarov Level 1

                Michael,

                 

                About  "out of sync" I mean "on one playback computer, audio and video were out of sync".

                 

                About your suggestion about setting multicast - I'll keep that in mind, but will continue to play Today I found one interesting thing while playing with window duration, I've set it to half of a second and published video to 2 clients, everything were very cool until one of clients disconnected - after that second client lost part of the publisher image - a lot of rectangles became grey and it couldn't return to good quality for a looong time, I guess there is some setting about that? Looks like it used retransmission of data through the disconnected peer and couldn't be reconfigured fast enough to get packets only from publisher when one viewer went away.

                • 5. Re: NetGroup multicast: delay, a/v out of sync
                  Michael Thornburgh Adobe Employee

                  with a half-second window duration, there isn't enough time for the viewer to get availability information about stream fragments from the publisher, decide it's heard from all the peers it's going to, and request the fragments it needs before the fragments are out-of-window.  most likely you were getting all of the fragments via push-mode, and the disconnected peer was in the push path between the remaining viewer and the publisher.  with that one peer gone and not enough time for pull-mode to get all the fragments, you'd have to wait until the push trees healed themselves to actually get the stream.  the multicast system builds and optimizes push trees relatively slowly to keep duplicated stream fragments to a minimum.

                   

                  if you set the window that short, you'll also need to turn down the availability update period and the fetch period such that their sum plus the round trip time between the peers (negligible if they're on the same LAN) is smaller than the window duration for pull-mode to have a hope of working.  and even then, with the window that short, there's very little margin to accommodate even the most minor network glitch, so quality will suffer.

                   

                  platform-dependent timer accuracy may limit or negatively impact the performance of the multicast system when any of these parameters are set to very short periods (less than 50msec/0.05sec on some platforms).  the shorter these periods, the more CPU time will be used on every peer in the mesh, and more network resources will be used for overhead as batching opportunities are lost.

                   

                  -mike

                  • 6. Re: NetGroup multicast: delay, a/v out of sync
                    mr suiss 83

                    Hello,

                     

                    I've been reading your post replies which are very interesting.

                    I've tested multicast example Michael provided with  (viewer.mxml/provider.mxml) and without changing NetStream.multicast* properties, I'm having the same kind of problem with a/v synchronization as aylarov.

                    I'm working on Windows 7 (with FireFox 3.6.3) and I've been testing streaming from Windows 7 my video to 2 friends ( PC (with FireFox) and Mac (with Safari) ), no virtual machines involved and no firewall, one was in the same room as mine and the other in a complete other location. I'm not expecting my friend (in the same room) to be completely synchronized with my local video/audio but we are experiencing the same problem, video is okay after half a second delay (which is fine for me) but audio takes, each time, about 20 seconds to synchronize with video.

                    I tried once putting NetStream.BufferTime to 0.0 before reading your posts, but didn't change anything. Then after reading your post, I deleted NetStream.BufferTime to 0.0 from my code and still experienced the same problem.

                    I've now installed the candidate release of FlashPlayer 10.1 (10.1.53.7) and asked my friends to do the same on Mac and Windows but still having a delay of 20 seconds for a good synchronization between Video and Audio.

                     

                    I've been looking how to reduce this delay (20 seconds to get Video/Audio synchronized is a reall problem for me) but without changing  NetStream.multicast* properties, I can't find any answers.

                     

                    By the way, I created a login panel and once authetificated, I initialize the application. Then I created an exit button which closes every opened stream, connection to Stratus2 server and brings you back to the login panel. When you try entering again (without reloading the web page), I go through all the initialization process as if it was the first time the application was loaded and synchronization between audio/video are great. It is really the first time the application is loaded and connections are created that I experience a problem of synchronization between A/V. Hope this will help.

                     

                    Sorry for my english. Hope you will reply to this post.

                    Sincerely

                    • 7. Re: NetGroup multicast: delay, a/v out of sync
                      Michael Thornburgh Adobe Employee

                      it sounds like your Windows 7 computer is the publisher.  what happens if the Mac or the other PC publishes?  are there still synchronization issues?  how about if you publish and subscribe on the same computer (different browser windows)?  does 1-1 P2P work well (you can use the VideoPhoneLabs live sample to test 1-1 communication and observe A/V sync)?

                       

                      -mike

                      • 8. Re: NetGroup multicast: delay, a/v out of sync
                        mr suiss 83 Level 1

                        Thanks for your quick answer. I'll be testing all what you said today and come back to you with my results.

                        Sincerely

                        • 9. Re: NetGroup multicast: delay, a/v out of sync
                          mr suiss 83 Level 1

                          Hi Mike,

                           

                          I'm back with more details. I tried publishing with a Mac instead of a Windows but still having the same problem. I tried out  VideoPhoneLabs live sample to test 1-1 communication and observe A/V  sync. Everything is working fine (but VideoPhoneLabs isn't using NetGroups multicast). I tried out publishing/subscribing on the same computer but doesn't make any difference. Still getting the same problem with audio so I tried out creating an application using only audio stream. Here are the details of my application:

                           

                          - I created a conference call application (uses only audio streaming) where all participants are allowed to speak and hear other participants.

                          - Here is how it works:

                               - First I create a NetConnection and connect to Stratus with "rtmfp://stratus.adobe.com/" and my developer key (Connection without out any problems).

                               - Once connected to the server, I create my GroupSpec with the following options:

                           

                                          _groupSpecifier = new GroupSpecifier( GROUP_NAME );
                                          _groupSpecifier.multicastEnabled = true;
                                          _groupSpecifier.objectReplicationEnabled = true;
                                          _groupSpecifier.postingEnabled = true;
                                          _groupSpecifier.routingEnabled = true;
                                           _groupSpecifier.serverChannelEnabled = true;
                                          _groupSpec = _groupSpecifier.groupspecWithoutAuthorizations();

                           

                               - Then I join the NetGroup and initialize my outgoingStream at the same time.

                               - Once "NetStream.Connect.Success" is dispatched, I start my outgoingStream:

                           

                                         - I attach the audio to the stream (I use "mic.codec = SoundCodec.SPEEX")

                                         - I publish it using "_outgoingStream.publish( _netGroup.convertPeerIDToGroupAddress(_nearID) )"

                           

                               - To listen to other participants stream, I listen to the event "NetGroup.MulticastStream.PublishNotify". Once this event is dispatched, I create as many incoming streams as participants joining the same group. If you need more details on how I create these incoming streams, I'll be happy to communicate them to you.

                           

                               - Everything goes fine, I can see all participants connected to the group and hear them all BUT here is my big problem. When participants are speaking there is an audio latency of 2-3 seconds before I can hear them (test based on "OK" written in SKYPE when a participant starts speaking). All participants have to wait about 20-30 seconds before a complete synchronization, meaning no latency on audio. Participants speak and are heard straight away. After these 20-30 seconds synchronization, communication is just amazing. Everything is perfect.

                           

                               - I tried this application on the following environment: every participants are publishing/subscribing using either Mac, Windows Vista, Windows 7 on either FireFox, Google Chrome or Safari. I tried it out as well just on my Windows 7 (publisher and subscriber) using several different browsers (to simulate other participants) but still getting the same problem (audio latency of 2-3 seconds and after 20-30 seconds, synchronization is perfect). Did the same again but just on one Mac but still the same problem.

                           

                          I hope my explainations are understandable. My goal is to reduce these 20-30 seconds before synchronization is good (no latency).

                          I think the problem is comming for the audio, because video is usually working great. It maybe comes from the fact I'm using NetGroup because 1-1 communication using VideoPhoneLabs (which is not using NetGroup... tell me if I'm wrong) and other application I've created before based on VideoPhoneLabs are working greate (no latency).

                           

                          I really can't get over this problem. Hope to hear from you soon.

                          Sincerely

                           

                          -suiss

                          • 10. Re: NetGroup multicast: delay, a/v out of sync
                            Michael Thornburgh Adobe Employee

                            a delay/latency of several seconds is expected with P2P multicast.  after 20-30 seconds you may see the latency go to nearly zero, but that performance can't be guaranteed as people join and leave the multicast group.  the maximum delay/jitter could be as much as the NetStream.multicastWindowDuration.  in a totally stable group, the multicast system will eventually create one or more minimum latency spanning trees through the group for each stream, but it takes time to build/optimize them (that's probably the 20-30 seconds you are seeing).

                             

                            if you are seeing the latency go to 0, that probably also means you have the NetStream.bufferTime set to 0 so there isn't any buffering.  you shouldn't set NetStream.bufferTime to 0 for multicast because there *will* be jitter, and without a buffer to absorb it the stream playback will be disrupted.

                             

                            we have not been able to replicate A/V out-of-sync in our labs, and A/V out-of-sync shouldn't happen with a native (not VM) OS & Flash Player installation.  your description above doesn't sound like A/V out-of-sync, although you have stated you observe that.  so to clarify: with a single multicast publisher and a single subscriber and the subscriber's NetStream.bufferTime > 0.0, does the subscriber see video synchronized with audio, even though there might be a multi-second delay between when-it-happened and when-you-see-and-hear-it?  example: someone says "1, 2, 3, 4" on camera, and maybe 3 seconds go by before it comes out the subscriber side, but when it does, you see the picture saying "1, 2, 3, 4" synchronized with hearing the "1, 2, 3, 4"?  or are the moving pictures separated from the associated audio by several seconds as they come out of the subscriber side?

                             

                            -mike

                            • 11. Re: NetGroup multicast: delay, a/v out of sync
                              mr suiss 83 Level 1

                              Thanks for your quick answer.

                              For the latency dropping nearly to 0 after 20-30 seconds isn't due to  "NetStream.bufferTime to 0 for multicast" as in your previous post you  told aylarov:

                              Michael Thornburgh wrote:

                              I recommend not changing the NetStream.multicast* properties

                              so I didn't change the bufferTime to 0 to avoid having other problems.

                              To answer your question about:

                              Michael Thornburgh wrote:

                               

                              your description above doesn't sound like A/V out-of-sync, although you have stated you observe that

                               

                              I had first created a project using Audio and Video with NetStream.multicast and indeed I had a problem of A/V out of sync meaning the moving pictures are separated from the  associated audio by several  seconds as they come out of the subscriber side. However, by setting a  very low camera mode "camera.setMode(80, 60, 15)", A/V are not anymore  out of synch, so I suppose it's a problem of brandwidth on my side. Then to localize the problem (because I believed it was an audio problem), I tried creating a project using only audio stream and NetStream.Multicast such as a ConferenceCall. But still had 20-30 seconds before latency drops near to 0.

                               

                              So at the end of the day, I had to problems:

                               

                                   - A/V out of synch -> Now it is fine as explained just above with "camera.setMode(80, 60, 15)"

                                   - 20-30 seconds before I see the latency go to nearly zero -> but now you have answered to this problem

                               

                              Michael Thornburgh wrote:

                               

                              in a totally stable group, the multicast system will eventually create one or more minimum latency spanning trees through the group for each stream, but it takes time to build/optimize them (that's probably the 20-30 seconds you are seeing).

                              Thank you very much for spending some time answering my question even though I wasn't very clear on my problem.

                              Sincerely

                               

                              -suiss

                              • 12. Re: NetGroup multicast: delay, a/v out of sync
                                Arvind Mehta Level 1

                                Hi Michael,

                                 

                                I read your whole comments and I also found such issue.

                                I limit my bandwidth to minimum value but the Netstream info give SRTT value nas audioLossRate = 0.

                                 

                                I dont know how to identify the latency.
                                Please help

                                 

                                Thank you

                                • 13. Re: NetGroup multicast: delay, a/v out of sync
                                  Michael Thornburgh Adobe Employee

                                  for a multicast/group NetStream, the SRTT has no meaning.  that is only applicable to client-server or DIRECT_CONNECTION NetStreams (and then, i believe the SRTT on the sender side would only be visible in the per-peer publishing NetStream -- that is, the members of the main publishing NetStream's peerStreams property).  i think the audioLossRate should also be 0 in the multicast case.  unfortunately, while RTMFP does internally calculate a "quality"

                                  of the received multicast stream, i don't see where that's exposed to ActionScript.

                                   

                                  you should be able to see how much is locally buffered by NetStream.bufferLength.