video latency on mobile device

[mriccobene]

I am working on an application that connects two peers in audio/video using rtmfp.

When a peer wants to talk with another, first it exchanges peer-id and stream-name with the other and then publishes its microphone/camera (in “live” mode) and plays the other peer stream. As usual in Flash platform. Message exchange in my application is server directed (I am using FMS 4), stream connection is peer-to-peer.

Now, all works fine when the two peer run on Adobe AIR for PC. But when one or both run on a mobile platform (iOS 4.3.3 for example) I experience a strange random latency. Sometime latency on the mobile side is near zero, sometime it grows to 3 even 5 or more seconds. So the communication is not really possible.

I tried to narrow down latency by testing different values of bufferTime both on incoming and on outgoing stream: bufferTime = 0 does not work fine on mobile side; using bufferTime = 0 in the output stream and bufferTime=0.3 on incoming stream works better.  With this values when video is not involved all works fine. But when one of the two peer attaches a camera then the result is not always the same: sometime latency is acceptable sometime it is not: bufferLength of the incoming stream (playing stream) grows to 3 or more seconds and then it goes up and down. Using bufferTimeMax do not improve latency. On the pc side bufferLength is near zero.

I noted that on the pc side sometime I got a NetStream.Buffer.Empty on the incoming stream so I am thinking that the problem is related to cpu resources on mobile side (I am using an iPhone 4). To lower resource consumption I set Camera.setQuality(0,100) and Camera.setMode(80,60,3). Microphone is configured to use Speex with encodeQuality = 6. But the problem remains.

Is the video playback of Flash Air on mobile device fast enough?

The two peer in my test are on the same LAN and the mobile device use Wi-Fi access. I use Flash CS 5.5 and AIR 2.7.

Does anyone have an explanation of this behavior?

Thanks

Michelangelo

[ch1f]

Hi! Do you have any success with you problem? I'm trying to solve same problem.

[mriccobene]

Hi,

after a month of tests I stopped looking for a solution.

In the last test I tried to split (desynchronize) audio stream and video stream so I had two different play buffers on which I tested different bufferTime values. I was hoping to have a low predictable latency on audio playback (that is more important for my application) and a different not specified latency on video. But my test failed: this solution works as the solution where audio and video have the same buffer and this is very revealing.

I don't know if this problem is related to:

1. a my error in configuring audio/video stream
2. a flash player bug (on iOS) in live mode
3. a high resource consumption of the Sorenson Spark codec implementation on AIR for iOS

I posted my initial question to learn from someone anything useful to distinguish among those alternatives. I tend towards the third.

Michelangelo

[mriccobene]

Hi,

now i am testing my application (see first post in this thread) with Air 3.0.

In the desktop-to-mobile or mobile-to-mobile (wi-fi) scenario the results are different:

- bufferTime = 0 --> audio has near zero latency but sometimes there is audio packet loss

- bufferTime > 0 --> latency has strange behaviour, it can't be controlled (bufferTimeMax doesn't work)

This behaviour is the same with video or without it.

So i cannot use bufferTime = 0 because there is audio packet loss,

and i cannot use bufferTime = 0.2 or 0.3 because latency grows up to many seconds!

The packets loss is strange: it happens even in a lan with wi-fi connection.

The same application works well in desktop-to-desktop scenario.

Any advice?

Michelangelo

[Joe ... Ward]

Please file a bug (http://bugbase.adobe.com).

Thanks,

Joe

[mriccobene · Accepted solution]

Hello,

i have reduced some sources of events (MovieClip with timeline -> Sprite, custom scroller, ...), brought back some changes made to avoid the problem of latency (video on a separate stream), and compiled using Air 3.1 and now everything work fine!

Using bufferTime = 0 i have got zero latency on Wi-Fi and 3G connections, in a mobile-to-mobile scenario!

Latency in the presence of video remains near zero on Wi-Fi connection, goes to 0.2 / 0.3 secs on 3G connection.

Sometime latency increases a little but then it decreases to zero; this happens regularly setting SoundMixer.useSpeakerphoneForVoice=true (i am investigating on this).

I made no test for the bufferTime > 0 case.

Best regards,

Michelangelo

[Guest]

Where are you setting bufferTime = 0 to achieve this?

Inside an event handler?

or on object create?

[mriccobene]

>Where are you setting bufferTime = 0 to achieve this?

On the NetStream object, before play():

var inputStream: NetStream = ...;

inputStream.bufferTime = 0;

inputStream.play(streamName);

Also on the output NetStream, but this is less important.