2 Replies Latest reply on Mar 17, 2009 7:15 PM by rohan mukkamala

    Audio Video synchronization issues


      I have a video application that I am building - and previously I used FMS and TCP connections for video. Now expecting the latency to be much lesser with UDP, I implemented UDP in my application and stratus as the RTMFP capable server. I used the sample video phone application's tutorial do a simple sendStream and receiveStream for video.

      However, unlike the hosted sample application itself, my audio and video are completely out of synchronization - about 4 secs!! I tried to look at the sample code to see if there were any special settings that were made. I could not find any.

      My questions:

      1. Did Adobe make any special tweaks in their hosted sample application? If so, what are they?
      2. Are there any known synchronization issues with using RTMFP connections for video?
      3. Does anyone else also face audio video synchronization issues?
      4. If none of the above is true, what do you think my mistakes are? Please let me know if I need to add more information to answer this question.

      Thanks in advance.
        • 1. Re: Audio Video synchronization issues
          Michael Thornburgh Adobe Employee
          with RTMFP, video and audio can go out of sync if the data rate of your stream exceeds your network capacity. video is currently sent with 100% reliability, so once it's queued for transmission, it's going to be sent (eventually). video is lower priority than audio, though, so your audio data gets first dibs on your network capacity. if the video stream's data rate is higher than what fits in your network (after audio), the video data will back up until its send buffer is filled, and new camera frames will stop getting captured. in the steady state, this can look like a multi-second offset between the video and audio.

          try turning your video rate/quality down so that it fits in your network capacity. video and audio should stay in sync.

          with RTMP, there's one network transmission buffer (TCP's) for all of the parts of your stream (audio & video). when you have insufficient network bandwidth, the TCP buffer will eventually fill up and video frames will stop being captured to compensate. so while audio and video might remain in sync, the total end-to-end latency will go up. when using RTMFP, the audio and video have independent transmission buffers, so in cases of insufficient network resources, the higher-priority audio should remain more timely but video may fall behind.

          • 2. Re: Audio Video synchronization issues
            Level 1
            Mike, thanks for the explanation. Really helps! I will try adjusting my video bit rate and check how it goes. In fact, for me sometimes AV were in sync and sometimes they go out of sync. With your reasoning I can see why ...

            Thanks a lot!