Getting lots of "NetStream.Play.InsufficientBW" when there's plenty of Bandwidth

[twelfth312]

Hi All,

We've upgraded our production servers to 3.5.3 on Windows 2008, and now I'm seeing a lot of "NetStream.Play.InsufficientBW" statuses in our client app's log.  The server is streaming a very low bandwidth live video/audio stream, and the client has more than plenty of bandwidth to play it.  The stream doesn't exhibit any audio drops or skipped frames, and there are no message drops on the server side.  We have had clients complaining of random disconnects.  Does anyone know why I might be getting this message?  If anyone from Adobe could help with this, I'd greatly appreciate it.  Thank you.

Best,

-Costin

[Janaki_Lakshmikanthan]

Hi,

Check whether your server is loaded so much and not able to send the data to the clients at the right rate. This can also lead to "Netstream.Play.InsufficientBW" message at the client side.

Regards,

Janaki L

[twelfth312]

Hi Janaki,

The server is definitely not overloaded.  It's a dedicated box with only 2 users connected.  Thanks.

-Costin

[MareleADI]

We had experience the same think.

Server was dedicated, 1 client , 350 kpbs stream and still the insufficientBW spam on client side.

ns.bufferTime = 3; solved that.

Ok we used 3 sec for a live recorded stream. If you are going with a live stream perhaps you need much less.

[twelfth312]

Well, its a live stream, and at one point I only had it streaming audio, at about 40kbps.  The client is on a T1, and I was looking at the network utilization and it was minimal.  The client definitely has sufficient bandwidth.  I did also apply the buffer, and it didn't resolve the issue.  Thanks for the help though.

-Costin

[Asa_-_FMS]

There's a known issue in FMS 3.5.3 that we're investigating that causes this.  For now you can work around the issue by disabling FMS Live Message Queueing until the patch released.  You can do so in your Application.xml - change the following config setting

Application/StreamManager/Live/Queue

change the enabled setting to "false"

Note that this workaround will increase your CPU usage somewhat, but restore proper buffering operations.  As mentioned before, we're aware of the issue and working on a fix right now.

Asa

[twelfth312]

Hi Asa,

Thanks for the reply! 

Unfortunately, I already had this in my Application.xml and it was still doing it:

     <Queue enabled="false">

Any other work arounds at this time?

In addition, I'm getting reports that users are getting disconnected.  After some arbitrary time, the server issues a "NetConnection.Connect.Rejected" for no reason, or at least none that I can find.  Is this also a known issue?  Please advise, since this is happening on our production systems.  Should I roll back to 3.5.2?

Thank you!

-Costin

[Asa_-_FMS]

Hmm, that workaround is pretty definitive to the issue I"m speaking of, so perhaps it's different.  I've got to get on top of that one for the time being, as it's an issue we need me to resolve right away.  So, I can offer to come back to this with you shortly to resolve, or try and guide you through some steps to confirm over the next few days.  LMK what's preferred?

Asa

[twelfth312]

Hi Asa,

Here's a quick update.  I removed 3.5.3 and reinstalled 3.5.2, and I'm still getting the exact same issue.

To give a little more context to this issue, I'm running the FMS 3.5.2 on Windows 2008 Server, Service Pack 2 on Amazon EC2.  The OS is 64-bit, with 7.5 GB of RAM and 2 E5430 @ 2.66 GHz processors.

I'm also randomly getting "NetConnection.Connect.Rejected" and users get disconnected from the app.

Any thoughts?

Thank you,

-Costin

[twelfth312]

One more update --

I installed 3.5.1 on the same Windows 2008 64-bit server, and its still doing the same thing.  For reference, we do not have this issue on a 3.5.1 box running Windows 2003 32-bit in the EC2 environment.  I'll keep posting whatever findings I make.  Thanks.

-Costin

[Janaki_Lakshmikanthan]

For frequent disconnect... Can you look up your access.log file for the status code that it has logged for those disconnected clients? To do so, you can go to $Root\Flash Media Server\logs\access.log. Search for the 'disconnect' events and its corresponding 'x-status' field. This can give us more hint to dig in theis issue.

Regards,

Janaki L

[twelfth312]

Hi Janaki,

These are all of the x-status fields:

103

200

422

423

427

429

I hope that helps.

Also, I installed 3.5.3 on Windows 2003 R2 64-bit, and now my issue has been resolved, so it seems to be related to Windows 2008.  I didn't try the 32 bit version of the OS, but it was on the 64 bit version that I had the problem.  Thanks.

-Costin

[Janaki_Lakshmikanthan]

Hi,

I looked at the status codes which you have mentioned. Thanks for the detailed information on the platforms in which you were running.

Could you provide us some more information to debug this issue at our end?

1. How much is the server loaded?

2. Can you reproduce this consistently?

3. What kind of stream those clients were playing. flv, mp4 or raw? on what bitrates?

4. Can you share your application which we can try in our local server?

Regards,

Janaki L

(Adobe Systems Inc)

[twelfth312]

Hi Janaki,

I've since switched back to Windows 2003 on Amazon EC2, and this specific problem has gone away.

However, I am getting lots of reports from users that they're getting audio stutterring, jerky video, etc.  I'm trying to figure out if this is an Amazon EC2 issue, a misconfiguration of the Application.xml, or some type of client side issue.  Is this something that Adobe Support can help me with?  Thank you.

Best,

-Costin