I have used BlazeDS in an application to push the data in realtime. However, we have been experiencing a heavy memory leak in the application. The application has a scheduler that pushes the data every one minute into the default queue. Without any browsers opened, the appication remain stable. And closing the browsers also cleans up the FlexClient object and other associated objects for that browsers. But the leak occurs when we open and keep browsers for a long period, say 2 to 3 days. The memory usage of application gradually increases and throws a "java.lang.OutOfMemoryError: Java heap space" Exception. On analyzing the HeapDump, I found that there are thousands of AsyncMessage Objects in the memory, not getting garbage-collected. This the channel definiton congured in services-config.xml file
<channel-definition id="my-polling-amf" class="mx.messaging.channels.AMFChannel">
<endpoint url="http://{server.name}:{server.port}/{context.root}/messagebroker/amfpolling" class="flex.messaging.endpoints.AMFEndpoint"/>
<properties>
<polling-enabled>true</polling-enabled>
<polling-interval-seconds>4</polling-interval-seconds>
<invalidate-session-on-disconnect>true</invalidate-session-on-disconn ect>
</properties>
</channel-definition>
Does anyone know the cause of this issue and how it can be overcome. Please help.
I found the same bug in my application , I profiled it and found out that there are lots of instance of PollCommandMessageResponder class in PollingChannel package , I obviously created one Subscriber and it continually create this instance ,
Adobe is so slow to fix these kind of bugs so I think emulate subscriber service myself.
You should certainly feel free to file a JIRA bug with a reproducible case. We will take a look at it. Note that a reproducible case involves more than the config - we need to have the code that makes the problem happen.
Or, since BlazeDS is open source, you could dig in to the code and provide a patch for the issue.
Higly reproductible with BlazeDS4 :
Simple application :
2 consumers.
Each consummer draws the result of the message (X/Y positionning from the message).
Each destination is StreamingAMF / JMS Adapter with ActiveMQ.
Launch clients (IE or FireFox) to consume the message.
Send no more than 50 messages/s on one of the destination only.
Press a few time F5.
you won't wait too long to get OOM.
Heapdump analysys shows :
264 970 instances of class flex.messaging.messages.AsyncMessage
The same test using BlazeDS 3.0 shows 0 instances of class flex.messaging.messages.AsyncMessage under the same load.
This first image shows a number of threads and a memory consumption always increasing - BlazeDS 4 behaviour.
(remember we press F5 to reload the application time to time)
This secondimage shows a number of threads aligned with the real number of clients - BlazeDS 3 behaviour.
No problems with th memory consumption.
(remember we press F5 to reload the application time to time)
HTH
Please file a bug with this information (and the repro code) in our bug tracking system - JIRA: http://bugs.adobe.com/blazeds/
Thanks
Tom
Hi.
I think the other people on this thread are talking about memory consumption on the server while it looks like you are talking about memory consumption on the client. It's quite normal when using a polling channel to see many instances off the PollCommandMessageResponder class getting created. One of these should get created for every poll request. For example, if your polling interval was 2 seconds you would see one of these getting created every two seconds or so.
Have you tried forcing garbage collection using the profiler? You should see these instances go away.
Hope that helps.
-Alex
Hi,
BlazeDS doesn't destroy the clients session right after the connection closed. The undeliverable messages are kept until the client timeout. The Flex Client can be restored if the same ID is used. In order to prevent high usage of memory, it should make the flex client timeout short. After the Flex Client timeout, the messages(AsyncMessage) are discarded. For the memory management, we can also have custom OutBoundQueueProcessor to limit the number of messages in the queue. (for the blazeds posting)
In short, the problem can be solved by setting the flex client timeout to 1 minute.
This is on server only. We are investigating the browser leak issue. We did some MTBF test, one of the JMS test used polling amf. We didn't see any memory issue in 7 days run. Can you let me know which build did you find the memory leak? Mohd.
William
BLZ-572 : created with some code/configuration to
reproduce.
I have tracked down (and fixed locally) a serious memory leak in PollingChannel.as. It turns out that PollCommandMessageResponder objects are getting created for every polling message but never getting released because they are getting held in memory by an event listener.
I filed a bug: https://bugs.adobe.com/jira/browse/SDK-27883
The fix is simple -- just cleanup the event listener. The nice thing about this bug is that you don't have to take my word for it -- please just look at PollingChannel.as code and notice at the bottom:
public function PollCommandMessageResponder(agent:MessageAgent, msg:IMessage, channel:PollingChannel, log:ILogger)
{
super(agent, msg, channel);
_log = log;
// Track channel connected state.
// If the channel disconnects while this poll is outstanding, suppress result/fault handling.
channel.addEventListener(PropertyChangeEvent.PROPERTY_CHANGE, channelPropertyChangeHandler);
}
Now look (in vain) to see where that event listener is cleaned up -- it isn't.
My fix is simple -- just release the event listener after it is no longer needed. I do it two places:
Place 1:
override protected function resultHandler(msg:IMessage):void
{
var pollingChannel:PollingChannel = channel as PollingChannel;
//
//VANTOS: This line was added to remove a serious memory leak
channel.removeEventListener(PropertyChangeEvent.PROPERTY_CHANGE, channelPropertyChangeHandler);
Place 2:
override protected function statusHandler(msg:IMessage):void
{
//
//VANTOS: This line was added to remove a serious memory leak
//
channel.removeEventListener(PropertyChangeEvent.PROPERTY_CHANGE, channelPropertyChangeHandler);
************
Please note that I've although I've verified that this does stop the memory leak (using the Flash Builder profiler), it is possible that there is a better way to fix this. But this fix seems to work for us.
I think there are two different issues here: possible memory leak on the server (BLZ-572) and the client (SDK-27883).
I haven't looked into this in too much detail but my guess is that BLZ-572 is probably not a bug. I think what's happening is that, the browser is closed (or the app is reloaded) with an HTTP based channel, messages accumulate until session times out which is expected. We've had similar bugs before where we explained people how to avoid it (i.e keep session timeout low, use disconnectAll trick etc.). I suggest you take a look the comments in this bug:
http://bugs.adobe.com/jira/browse/BLZ-502
Regardless, we'll take a look at BLZ-572 to see if it's different from BLZ-502.
In SDK-27883, there's a valid a point that we should remove the event listener at some point but I don't think not removing it will necessarily cause a memory leak as indicated, need to verify it with profiler to make sure. My guess is it doesn't cause any memory leak but I agree that we should remove the event listener to be on the safe side.
Yes, I agree these do appear to be two separate issues. Thank you for clearly separating them.
There really is a memory leak on the client. If you have any doubts please run the Flash Profiler and watch the counts for PollCommandMessageResponder (you will need to set up a polling connection of course).
Did you get a chance to look at the code that I pointed out in PollingChannel.as? You can see that for each poll a PollCommandMessageResponder is getting created. And each one of those registers an event listener in its constructor. That means Flash has to keep that PollCommandMessageResponder in memory. I don't see any way around that, but perhaps I'm missing something here? I'm asking that with honest curiosity.
By the way, if I'm right it actually is worse than just a memory leak -- if we did happen to get the error event that all those PollCommandMessageResponder are listening for, all of them would suddenly have to be called which could be quite a large amount of work that might explain some of the random crashes we've seen.
Hi Everyone,
First of all thank you very much for all your replies. Due to busy work schedule, I could not reply to this thread.
My memory leak problem is still there. In fact, it has still got worsen. The application is getting crashed/hanged 2 to 3 times everyday.
I have created a bug in JIRA bug at http://bugs.adobe.com/jira/browse/BLZ-585.
I hope I have furnished enough information to reproduce the bug. In case, if you need anything more, please let me know.
Looking forward for early solution.
Hi All,
I am facing the same issue while using the BlazeDS and Spring flex Jars .I have tried all the possible solutions :-
1) disconnect all in SWF
2) Invalidate session on disconnect in serviceconfig.xml
3) session time out in flex configuration file (serviceconfig.xml ) and web.xml --Reduce to 1 -2 minutes
But still getting the growing number of instance of (flex.messaging.messages.AsyncMessage) and the objects which is a part of the body of the Asyn message when the browser is closed .
It seems that there is no acknowledgment from the flex session to stop messages from the BlazeDS
Flex session is alive which causes the server to keep pushing the messages.
I need help to overcome this issue
Please also mention the version of spring -flex and Blaze DS to be used .Is that version is available in Maven Central repository .
Thanks,
Rahul S
Hi Rahul,
Have you got a chance to follow the issues mentioned in the post above? I assume those did not help
Please open a JIRA bug with the following information:
1. Environment details.
2. A standalone reproducible end-to-end test scenario that illustrates your problem.
BlazeDS 4.6 requires Spring Flex 1.5.2. No, at this point, we don’t push the BlazeDS binaries in the Maven repository.
Rohit
North America
Europe, Middle East and Africa
Asia Pacific