Cold Fusion MX 7 is very slow! Help!

Report · Jul 25, 2006

We have been having a problem where the server occasionally running very slow or simply hanging. We have to restart the application server to get it up and running again. This happens few times a day, and I am not sure how to fix it. I got some thread dumps, but couldn't really make sense of it. Any help is greatly appreciated! By the way, I am running a single CPU Windows 2003 Server, SQL DB, and MX 7.

Below are the thread dumps during the non-response time.

Thread Stack TraceTrace Time: 17:21:35.171 25-Jul-2006 Request ID: 5775
Script Name: http://www.mysite.com/index.cfm
Started: 17:19:27.765 25-Jul-2006Exec Time: 127406msMemory Used: (12%)64,914KBMemory Free: 439,981KBThread ID: jrpp-103Priority: 5Hashcode: 10290709"jrpp-103" prio=5 tid=0x040948a8 nid=0x13cc runnable [e75f000..e75fdb8]
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.net.SocketOutputStream.write(Unknown Source)
at jrun.servlet.jrpp.JrppOutputStream$SpillStream.write(JrppOutputStream.java:182)
at jrun.servlet.io.MetricsOutputStream.write(MetricsOutputStream.java:75)
at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
at java.io.BufferedOutputStream.flush(Unknown Source)
- locked <0x150cad48> (a jrun.servlet.jrpp.JrppBufferedOutputStream)
at jrun.servlet.jrpp.JrppOutputStream.flush(JrppOutputStream.java:45)
at com.intergral.fusionreactor.filter.A.E.flush(Unknown Source)
at sun.nio.cs.StreamEncoder$CharsetSE.implFlush(Unknown Source)
at sun.nio.cs.StreamEncoder.flush(Unknown Source)
- locked <0x156285c8> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush(Unknown Source)
at java.io.PrintWriter.flush(Unknown Source)
- locked <0x156285c8> (a java.io.OutputStreamWriter)
at coldfusion.runtime.NeoJspWriter.flush(NeoJspWriter.java:210)
at coldfusion.runtime.NeoPageContext.flushOutput(NeoPageContext.java:1626)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:42)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.filter.RequestThrottleFilter.invoke(RequestThrottleFilter.java:115)
at coldfusion.CfmServlet.service(CfmServlet.java:107)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:78)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at com.intergral.fusionreactor.filter.FusionReactorFilter.B(Unknown Source)
at com.intergral.fusionreactor.filter.FusionReactorFilter.A(Unknown Source)
at com.intergral.fusionreactor.filter.FusionReactorFilter.doFilter(Unknown Source)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:91)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:257)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:541)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:204)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:318)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:426)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:264)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

================================

Thread Stack TraceTrace Time: 17:23:05.125 25-Jul-2006Request ID: 5884Script Name: http://www.mysite.com/moreinfo.cfm?Product_ID=10118Started: 17:22:25.31 25-Jul-2006Exec Time: 40094msMemory Used: (7%)36,112KBMemory Free: 468,783KBThread ID: jrpp-141Priority: 5Hashcode: 27784963"jrpp-141" prio=5 tid=0x04d7aa38 nid=0x1e50 runnable [3c60f000..3c60fdb8]
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.net.SocketOutputStream.write(Unknown Source)
at jrun.servlet.jrpp.JrppOutputStream$SpillStream.write(JrppOutputStream.java:182)
at jrun.servlet.io.MetricsOutputStream.write(MetricsOutputStream.java:75)
at java.io.BufferedOutputStream.write(Unknown Source)
- locked <0x153d4738> (a jrun.servlet.jrpp.JrppBufferedOutputStream)
at jrun.servlet.jrpp.JrppOutputStream.write(JrppOutputStream.java:70)
at com.intergral.fusionreactor.filter.A.E.write(Unknown Source)
at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes(Unknown Source)
at sun.nio.cs.StreamEncoder$CharsetSE.implWrite(Unknown Source)
at sun.nio.cs.StreamEncoder.write(Unknown Source)
- locked <0x1578d700> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.write(Unknown Source)
at java.io.PrintWriter.write(Unknown Source)
- locked <0x1578d700> (a java.io.OutputStreamWriter)
at java.io.CharArrayWriter.writeTo(Unknown Source)
- locked <0x15715ac8> (a coldfusion.runtime.CharBuffer)
at coldfusion.runtime.NeoJspWriter.flush(NeoJspWriter.java:208)
at coldfusion.runtime.NeoPageContext.flushOutput(NeoPageContext.java:1626)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:42)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.filter.RequestThrottleFilter.invoke(RequestThrottleFilter.java:115)
at coldfusion.CfmServlet.service(CfmServlet.java:107)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:78)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at com.intergral.fusionreactor.filter.FusionReactorFilter.B(Unknown Source)
at com.intergral.fusionreactor.filter.FusionReactorFilter.A(Unknown Source)
at com.intergral.fusionreactor.filter.FusionReactorFilter.doFilter(Unknown Source)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:91)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:257)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:541)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:204)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:318)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:426)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:264)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

=======================================================

Thread Stack TraceTrace Time: 17:30:23.984 25-Jul-2006Request ID: 6089Script Name: http://www.mysite.com/category.cfm?Category=249Started: 17:27:59.46 25-Jul-2006Exec Time: 144938msMemory Used: (12%)63,877KBMemory Free: 441,018KBThread ID: jrpp-139Priority: 5Hashcode: 1071843"jrpp-139" prio=5 tid=0x041609f8 nid=0x15c0 runnable [3c40f000..3c40fdb8]
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.net.SocketOutputStream.write(Unknown Source)
at jrun.servlet.jrpp.JrppOutputStream$SpillStream.write(JrppOutputStream.java:182)
at jrun.servlet.io.MetricsOutputStream.write(MetricsOutputStream.java:75)
at java.io.BufferedOutputStream.write(Unknown Source)
- locked <0x152b5470> (a jrun.servlet.jrpp.JrppBufferedOutputStream)
at jrun.servlet.jrpp.JrppOutputStream.write(JrppOutputStream.java:70)
at com.intergral.fusionreactor.filter.A.E.write(Unknown Source)
at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes(Unknown Source)
at sun.nio.cs.StreamEncoder$CharsetSE.implWrite(Unknown Source)
at sun.nio.cs.StreamEncoder.write(Unknown Source)
- locked <0x1583c700> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.write(Unknown Source)
at java.io.PrintWriter.write(Unknown Source)
- locked <0x1583c700> (a java.io.OutputStreamWriter)
at java.io.CharArrayWriter.writeTo(Unknown Source)
- locked <0x156b8230> (a coldfusion.runtime.CharBuffer)
at coldfusion.runtime.NeoJspWriter.flush(NeoJspWriter.java:208)
at coldfusion.runtime.NeoPageContext.flushOutput(NeoPageContext.java:1626)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:42)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.filter.RequestThrottleFilter.invoke(RequestThrottleFilter.java:115)
at coldfusion.CfmServlet.service(CfmServlet.java:107)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:78)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at com.intergral.fusionreactor.filter.FusionReactorFilter.B(Unknown Source)
at com.intergral.fusionreactor.filter.FusionReactorFilter.A(Unknown Source)
at com.intergral.fusionreactor.filter.FusionReactorFilter.doFilter(Unknown Source)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:91)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:257)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:541)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:204)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:318)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:426)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:264)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

Any pointers/help are greatly appreciated!

Thanks!

Report · Sep 07, 2007

We've been having a similar issue, with occasional pages hanging for hundreds of seconds at that same JrppBufferedOutputStream, which SeeFusion identifies as "Writing completed page back to the Web server".

This happens randomly throughout the day, more often when there's more traffic, but also in the wee hours with relatively few visitors; nothing else appears to be going on, often with the long-running thread the only active thread.

Our setup is pretty similar to yours, CFMX7, MSSQL2000, Win2k3 Server, but with dual Xeons instead of single-CPU. We've applied the significant hotfixes, including the DST update to the 1.4.2_11 JVM. I'm not positive we've updated to the latest IIS connector (as mentioned in http://www.adobe.com/support/security/bulletins/apsb07-02.html), but will be checking, just to be sure. If you also don't have the newest connector installed, that would be an interesting commonality.

We had problems in the past with CF locking up completely during one of these hung threads, but after some tweaking to stabilize memory usage below 80%, the lockups are rare.

So I have no answer, but will be sure to share any solution we do find with you, and hope you'll do the same! :*)

Adobe Community

Cold Fusion MX 7 is very slow! Help!