1 Reply Latest reply on Nov 1, 2013 8:59 AM by DCwebGuy

    CF10 server monitor UI data collection keeps stalling every few mintues

    DCwebGuy Level 1

      I recently started running CF10 on a totally clean Windows 2008 machine (VM) running IIS 7.5.  I set everything up.  On my old machine (non-vm) running CF9, Windows 2003, and IIS6 I never had trouble with the server monitor interface.  Now, the interface will stop reporting every few mintues, and does not start tracking again unless I tab into one of the other windows and click some links then go back to the main interface.  I have tried every web browser, and on multiple machines to make sure it's not a client issue, and it's not as far as I can tell.

       

      I even added my vote to the bug database here, which may or may not be related: https://bugbase.adobe.com/index.cfm?event=bug&id=3539667

       

      I do not use internal cache to store queries, nor do I want to.

       

      The ColdFusion application log file is no help because the rows are empty where they correspond to the times when the monitor UI stops tracking (see attached screen shot).  ColdFusion-out.log does, however, provide the info below at each of the time frames that the UI stopped collecting data, but I can't find much in the way of a solution with this data:

       

      Oct 29, 2013 11:00:47 AM Error [qtp2093217297-3844] - 
      [BlazeDS]Exception when invoking service 'remoting-service': flex.messaging.MessageException: java.util.ConcurrentModificationException
        incomingMessage: Flex Message (flex.messaging.messages.RemotingMessage) 
          operation = getRealtimeStats
          clientId = 2D3F0114-690D-B4F0-6E11-CD0F3CDA72AC
          destination = ColdFusion
          messageId = 65D6531E-5559-ED6F-9435-04BB35AA7649
          timestamp = 1383058847216
          timeToLive = 0
          body = 
          [
      
          ]
          hdr(DSRequestTimeout) = 30
          hdr(DSId) = 2D3EFF72-EE1C-D8BF-F44D-256960FBA857
          hdr(DSEndpoint) = my-cfamf
        Exception: flex.messaging.MessageException: java.util.ConcurrentModificationException
          at coldfusion.flash.messaging.ColdFusionAdapter.getFlexError(ColdFusionAdapter.java:472)
          at coldfusion.flash.messaging.ColdFusionAdapter.invoke(ColdFusionAdapter.java:366)
          at flex.messaging.services.RemotingService.serviceMessage(RemotingService.java:183)
          at flex.messaging.MessageBroker.routeMessageToService(MessageBroker.java:1400)
          at flex.messaging.endpoints.AbstractEndpoint.serviceMessage(AbstractEndpoint.java:1005)
          at flex.messaging.endpoints.amf.MessageBrokerFilter.invoke(MessageBrokerFilter.java:103)
          at flex.messaging.endpoints.amf.LegacyFilter.invoke(LegacyFilter.java:158)
          at flex.messaging.endpoints.amf.SessionFilter.invoke(SessionFilter.java:44)
          at flex.messaging.endpoints.amf.BatchProcessFilter.invoke(BatchProcessFilter.java:67)
          at flex.messaging.endpoints.amf.SerializationFilter.invoke(SerializationFilter.java:166)
          at flex.messaging.endpoints.BaseHTTPEndpoint.service(BaseHTTPEndpoint.java:291)
          at coldfusion.flash.messaging.CFAMFEndPoint.service(CFAMFEndPoint.java:308)
          at flex.messaging.MessageBrokerServlet.service(MessageBrokerServlet.java:353)
          at coldfusion.flex.ColdFusionMessageBrokerServlet.service(ColdFusionMessageBrokerServlet.java:114)
          at coldfusion.monitor.jetty.server.MonitoringServlet.handleFlashRemotingReq(MonitoringServlet.java:356)
          at coldfusion.monitor.jetty.server.MonitoringServlet.service(MonitoringServlet.java:103)
          at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
          at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:530)
          at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:426)
          at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:118)
          at org.eclipse.jetty.server.session.SessionHandler.handle(SessionHandler.java:179)
          at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:930)
          at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:361)
          at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:864)
          at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:116)
          at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
          at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:112)
          at org.eclipse.jetty.server.Server.handle(Server.java:337)
          at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:552)
          at org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:993)
          at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:739)
          at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:209)
          at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:399)
          at org.eclipse.jetty.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:437)
          at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:435)
          at java.lang.Thread.run(Thread.java:722)
      

       

       

      I have tried everything from disabling memory and profile tracking, to rebooting my server (this is production by the way), and I have all the latest CF10 updates applied.  Why is the server/performance monitor it doing this?  I like to leave it up and running on a separate screen so I can check it periodically throughout the day, now I can't watch it for more than a few mins before it needs attention.  It's like it just stalls.  The UI itself is not locked-up, it simply stops graphing any data.

       

      Anyone else seen this or experienced the problem?  Thanks for any ideas.

       

      P.S.  Here is what application.log shows if it helps.  You can see the timing, and how one of those entries correlates to the log data above.

       

      Capture.JPG