• Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
    Dedicated community for Japanese speakers
  • 한국 커뮤니티
    Dedicated community for Korean speakers
Exit
0

Server hanging httpd threads & eventually creashing

Participant ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

I think my coldfusion server is doing somehting odd, it appears to be hanging on to httpd processess & not letting them go - ventually running the server out of memory

I get this when I try to stop the server :

Stopping ColdFusion 8, please wait
Error occurred during initialization of VM
Could not reserve enough space for object heap
*** glibc detected *** /opt/coldfusion8/runtime/bin/coldfusion8: free(): invalid pointer: 0xbffe9660 ***
======= Backtrace: =========
/lib/libc.so.6[0xb7ee6aa6]
/lib/libc.so.6(cfree+0x90)[0xb7ee9fc0]
/opt/coldfusion8/runtime/jre/lib/i386/server/libjvm.so[0x6602f93]
/opt/coldfusion8/runtime/jre/lib/i386/server/libjvm.so[0x65b3fe0]
/opt/coldfusion8/runtime/jre/lib/i386/server/libjvm.so[0x65b5c1d]
/opt/coldfusion8/runtime/jre/lib/i386/server/libjvm.so[0x65b9099]
/opt/coldfusion8/runtime/jre/lib/i386/server/libjvm.so(JNI_CreateJavaVM+0x53)[0x639c293]
/opt/coldfusion8/runtime/bin/coldfusion8[0x804c518]
/opt/coldfusion8/runtime/bin/coldfusion8[0x804ad7b]
/opt/coldfusion8/runtime/bin/coldfusion8[0x804a0f2]
/opt/coldfusion8/runtime/bin/coldfusion8[0x8049036]
/lib/libc.so.6(__libc_start_main+0xdc)[0xb7e95dec]
/opt/coldfusion8/runtime/bin/coldfusion8(dlopen+0x41)[0x8048ea1]
======= Memory map: ========
06000000-0665c000 r-xp 00000000 00:59 86672232   /opt/coldfusion8/runtime/jre/lib/i386/server/libjvm.so
0665c000-066a0000 rwxp 0065c000 00:59 86672232   /opt/coldfusion8/runtime/jre/lib/i386/server/libjvm.so
066a0000-06ac2000 rwxp 066a0000 00:00 0
08048000-08051000 r-xp 00000000 00:59 86682252   /opt/coldfusion8/runtime/bin/coldfusion8
08051000-08052000 rwxp 00008000 00:59 86682252   /opt/coldfusion8/runtime/bin/coldfusion8
08052000-08095000 rwxp 08052000 00:00 0
80400000-80421000 rwxp 80400000 00:00 0
80421000-80500000 --xp 80421000 00:00 0
8050d000-80518000 r-xp 00000000 00:59 25170338   /lib/libgcc_s-4.1.2-20070626.so.1
80518000-80519000 rwxp 0000a000 00:59 25170338   /lib/libgcc_s-4.1.2-20070626.so.1
b4d22000-b4d2b000 rwxp b4d22000 00:00 0
b4d2b000-b4de2000 rwxp b4d2b000 00:00 0
b4de2000-b5022000 rwxp b4de2000 00:00 0
b5022000-b7de2000 rwxp b5022000 00:00 0
b7de2000-b7df1000 r-xp 00000000 00:59 86672238   /opt/coldfusion8/runtime/jre/lib/i386/libzip.so
b7df1000-b7df3000 rwxp 0000e000 00:59 86672238   /opt/coldfusion8/runtime/jre/lib/i386/libzip.so
b7df3000-b7e16000 r-xp 00000000 00:59 86672272   /opt/coldfusion8/runtime/jre/lib/i386/libjava.so
b7e16000-b7e18000 rwxp 00023000 00:59 86672272   /opt/coldfusion8/runtime/jre/lib/i386/libjava.so
b7e18000-b7e23000 r-xp 00000000 00:59 86672264   /opt/coldfusion8/runtime/jre/lib/i386/libverify.so
b7e23000-b7e24000 rwxp 0000b000 00:59 86672264   /opt/coldfusion8/runtime/jre/lib/i386/libverify.so
b7e24000-b7e2d000 r-xp 00000000 00:59 25170318   /lib/libnss_files-2.5.so
b7e2d000-b7e2e000 r-xp 00008000 00:59 25170318   /lib/libnss_files-2.5.so
b7e2e000-b7e2f000 rwxp 00009000 00:59 25170318   /lib/libnss_files-2.5.so
b7e2f000-b7e42000 r-xp 00000000 00:59 25170304   /lib/libnsl-2.5.so
b7e42000-b7e43000 r-xp 00012000 00:59 25170304   /lib/libnsl-2.5.so
b7e43000-b7e44000 rwxp 00013000 00:59 25170304   /lib/libnsl-2.5.so
b7e44000-b7e46000 rwxp b7e44000 00:00 0
b7e47000-b7e4f000 rwxs 00000000 68:05 17126466   (deleted) /tmp/hsperfdata_apache/30648
b7e4f000-b7e56000 r-xp 00000000 00:59 25170246   /lib/librt-2.5.so
b7e56000-b7e57000 r-xp 00006000 00:59 25170246   /lib/librt-2.5.so
b7e57000-b7e58000 rwxp 00007000 00:59 25170246   /lib/librt-2.5.so
b7e58000-b7e7d000 r-xp 00000000 00:59 25170336   /lib/libm-2.5.so
b7e7d000-b7e7e000 r-xp 00024000 00:59 25170336   /lib/libm-2.5.so
b7e7e000-b7e7f000 rwxp 00025000 00:59 25170336   /lib/libm-2.5.so
b7e7f000-b7e80000 rwxp b7e7f000 00:00 0
b7e80000-b7fba000 r-xp 00000000 00:59 25170592   /lib/libc-2.5.so
b7fba000-b7fbc000 r-xp 0013a000 00:59 25170592   /lib/libc-2.5.so
b7fbc000-b7fbd000 rwxp 0013c000 00:59 25170592   /lib/libc-2.5.so
b7fbd000-b7fc0000 rwxp b7fbd000 00:00 0
b7fc0000-b7fc2000 r-xp 00000000 00:59 25170292   /lib/libdl-2.5.so
b7fc2000-b7fc3000 r-xp 00001000 00:59 25170292   /lib/libdl-2.5.so
b7fc3000-b7fc4000 rwxp 00002000 00:59 25170292   /lib/libdl-2.5.so
b7fc4000-b7fd7000 r-xp 00000000 00:59 25170430   /lib/libpthread-2.5.so
b7fd7000-b7fd8000 r-xp 00012000 00:59 25170430   /lib/libpthread-2.5.so
b7fd8000-b7fd9000 rwxp 00013000 00:59 25170430   /lib/libpthread-2.5.so
b7fd9000-b7fdc000 rwxp b7fd9000 00:00 0
b7fdc000-b7fe2000 r-xp 00000000 00:59 86672314   /opt/coldfusion8/runtime/jre/lib/i386/native_threads/libhpi.so
b7fe2000-b7fe3000 rwxp 00006000 00:59 86672314   /opt/coldfusion8/runtime/jre/lib/i386/native_threads/libhpi.so
b7fe3000-b7fe4000 rwxp b7fe3000 00:00 0
b7fe4000-b7fe5000 r-xp b7fe4000 00:00 0
b7fe5000-b7ffe000 r-xp 00000000 00:59 25170244   /lib/ld-2.5.so
b7ffe000-b7fff000 r-xp 00019000 00:59 25170244   /lib/ld-2.5.so
b7fff000-b8000000 rwxp 0001a000 00:59 25170244   /lib/ld-2.5.so
bffae000-bffb1000 --xp bffae000 00:00 0
bffb1000-bfffe000 rwxp bffb1000 00:00 0
ffffe000-fffff000 r-xp 00000000 00:00 0
sh: line 1: 30648 Aborted                 /opt/coldfusion8/runtime/bin/coldfusion8 -jar jrun.jar stop coldfusion
The ColdFusion 8 server seems to be hanging, will stop non-gracefully
ColdFusion 8 has been stopped

any help? ideas? getting desperate

-thanks

-sean

Views

4.3K

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

Hi Sean,

You do not mention the platform. From the error I guess Unix/Linux. Is that 32bit? Perhaps you have configured too much memory to the Minimum or Maximum heap and crossed the 32 bit memory limit threshold. What are your JVM arguments for Xmx Xms ?

HTH, Carl.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

Hi;

sorry - in a bit of a blind panic right now ....

it's a MediaTemple VM 4G memory running on CentOS 5.something.

the jvm args are :

-server -Dsun.io.useCanonCaches=false -XX:MaxPermSize=192m -XX:+UseParallelGC -Dcoldfusion.rootDir={application.home}/../ -Dcoldfusion.libPath={application.home}/../lib

the max heap is 512m minimum is blank

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

I also get this error - but only on a few different pages - if I restart CF a couple of times - it goes away.

500

ROOT CAUSE: 
coldfusion.runtime.EventHandlerException: Event handler exception.
     at coldfusion.runtime.AppEventInvoker.onRequest(AppEventInvoker.java:251)
     at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:278)
     at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
     at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
     at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
     at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
     at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:74)
     at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)
     at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
     at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
     at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
     at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
     at coldfusion.CfmServlet.service(CfmServlet.java:198)
     at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
     at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
     at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42)
     at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
     at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
     at jrun.servlet.FilterChain.service(FilterChain.java:101)
     at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
     at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
     at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
     at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
     at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
     at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
     at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
Caused by: java.lang.ExceptionInInitializerError
     at org.apache.bcel.util.SyntheticRepository.<clinit>(SyntheticRepository.java:86)
     at org.apache.bcel.classfile.JavaClass.<init>(JavaClass.java:110)
     at org.apache.bcel.classfile.JavaClass.<init>(JavaClass.java:229)
     at org.apache.bcel.generic.ClassGen.getJavaClass(ClassGen.java:230)
     at coldfusion.bytecode.JavaAssembler.getBytes(JavaAssembler.java:379)
     at coldfusion.compiler.EvaluateAssembler.assembleSet(EvaluateAssembler.java:87)
     at coldfusion.compiler.ExprClassLoader.defineSet(ExprClassLoader.java:44)
     at coldfusion.compiler.ExprClassLoader$SetCache.fetch(ExprClassLoader.java:100)
     at coldfusion.util.LruCache.get(LruCache.java:180)
     at coldfusion.compiler.ExprClassLoader$ExprCache.fetchSerial(ExprClassLoader.java:362)
     at coldfusion.util.AbstractCache.fetch(AbstractCache.java:58)
     at coldfusion.util.SoftCache.get(SoftCache.java:81)
     at coldfusion.compiler.ExprClassLoader.set(ExprClassLoader.java:256)
     at coldfusion.runtime.CFPage.SetVariable(CFPage.java:3922)
     at coldfusion.runtime.CfJspPage._checkParam(CfJspPage.java:2910)
     at coldfusion.tagext.lang.ParamTag.doStartTag(ParamTag.java:138)
     at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
     at cffunctions2ecfm1152816649$funcFIELDSDEFAULTS.runFunction(/var/www/vhosts/newride.ca/subdomains/admin/httpdocs/Assets/CFC/functions.cfm:222)
     at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:418)
     at coldfusion.runtime.UDFMethod$ReturnTypeFilter.invoke(UDFMethod.java:360)
     at coldfusion.runtime.UDFMethod$ArgumentCollectionFilter.invoke(UDFMethod.java:324)
     at coldfusion.filter.FunctionAccessFilter.invoke(FunctionAccessFilter.java:59)
     at coldfusion.runtime.UDFMethod.runFilterChain(UDFMethod.java:277)
     at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:192)
     at coldfusion.runtime.CfJspPage._invokeUDF(CfJspPage.java:2471)
     at cfmanage2dinventory2dedit2ecfm24489901.runPage(/var/www/vhosts/newride.ca/subdomains/admin/httpdocs/Assets/XHTML/manage-inventory-edit.cfm:98)
     at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
     at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:483)
     at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
     at cfindex2ecfm1182266435.runPage(/var/www/vhosts/newride.ca/subdomains/admin/httpdocs/index.cfm:27)
     at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
     at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:483)
     at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
     at cfApplication2ecfc1506803272$funcONREQUEST.runFunction(/var/www/vhosts/newride.ca/subdomains/admin/httpdocs/Application.cfc:126)
     at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:418)
     at coldfusion.runtime.UDFMethod$ArgumentCollectionFilter.invoke(UDFMethod.java:324)
     at coldfusion.filter.FunctionAccessFilter.invoke(FunctionAccessFilter.java:59)
     at coldfusion.runtime.UDFMethod.runFilterChain(UDFMethod.java:277)
     at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:192)
     at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:448)
     at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:308)
     at coldfusion.runtime.AppEventInvoker.invoke(AppEventInvoker.java:74)
     at coldfusion.runtime.AppEventInvoker.onRequest(AppEventInvoker.java:243)
     ... 27 more
Caused by: java.security.AccessControlException: access denied (java.io.FilePermission /opt/coldfusion8/runtime/servers/lib read)
     at java.security.AccessControlContext.checkPermission(AccessControlContext.java:323)
     at java.security.AccessController.checkPermission(AccessController.java:546)
     at java.lang.SecurityManager.checkPermission(SecurityManager.java:532)
     at java.lang.SecurityManager.checkRead(SecurityManager.java:871)
     at java.io.File.exists(File.java:731)
     at org.apache.bcel.util.ClassPath.getPathComponents(ClassPath.java:142)
     at org.apache.bcel.util.ClassPath.getClassPath(ClassPath.java:160)
     at org.apache.bcel.util.ClassPath.<init>(ClassPath.java:113)
     at org.apache.bcel.util.ClassPath.<clinit>(ClassPath.java:69)
     ... 70 more

javax.servlet.ServletException: ROOT CAUSE: 
coldfusion.runtime.EventHandlerException: Event handler exception.
     at coldfusion.runtime.AppEventInvoker.onRequest(AppEventInvoker.java:251)
     at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:278)
     at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
     at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
     at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
     at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
     at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:74)
     at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)
     at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
     at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
     at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
     at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
     at coldfusion.CfmServlet.service(CfmServlet.java:198)
     at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
     at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
     at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42)
     at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
     at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
     at jrun.servlet.FilterChain.service(FilterChain.java:101)
     at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
     at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
     at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
     at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
     at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
     at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
     at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
Caused by: java.lang.ExceptionInInitializerError
     at org.apache.bcel.util.SyntheticRepository.<clinit>(SyntheticRepository.java:86)
     at org.apache.bcel.classfile.JavaClass.<init>(JavaClass.java:110)
     at org.apache.bcel.classfile.JavaClass.<init>(JavaClass.java:229)
     at org.apache.bcel.generic.ClassGen.getJavaClass(ClassGen.java:230)
     at coldfusion.bytecode.JavaAssembler.getBytes(JavaAssembler.java:379)
     at coldfusion.compiler.EvaluateAssembler.assembleSet(EvaluateAssembler.java:87)
     at coldfusion.compiler.ExprClassLoader.defineSet(ExprClassLoader.java:44)
     at coldfusion.compiler.ExprClassLoader$SetCache.fetch(ExprClassLoader.java:100)
     at coldfusion.util.LruCache.get(LruCache.java:180)
     at coldfusion.compiler.ExprClassLoader$ExprCache.fetchSerial(ExprClassLoader.java:362)
     at coldfusion.util.AbstractCache.fetch(AbstractCache.java:58)
     at coldfusion.util.SoftCache.get(SoftCache.java:81)
     at coldfusion.compiler.ExprClassLoader.set(ExprClassLoader.java:256)
     at coldfusion.runtime.CFPage.SetVariable(CFPage.java:3922)
     at coldfusion.runtime.CfJspPage._checkParam(CfJspPage.java:2910)
     at coldfusion.tagext.lang.ParamTag.doStartTag(ParamTag.java:138)
     at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
     at cffunctions2ecfm1152816649$funcFIELDSDEFAULTS.runFunction(/var/www/vhosts/newride.ca/subdomains/admin/httpdocs/Assets/CFC/functions.cfm:222)
     at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:418)
     at coldfusion.runtime.UDFMethod$ReturnTypeFilter.invoke(UDFMethod.java:360)
     at coldfusion.runtime.UDFMethod$ArgumentCollectionFilter.invoke(UDFMethod.java:324)
     at coldfusion.filter.FunctionAccessFilter.invoke(FunctionAccessFilter.java:59)
     at coldfusion.runtime.UDFMethod.runFilterChain(UDFMethod.java:277)
     at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:192)
     at coldfusion.runtime.CfJspPage._invokeUDF(CfJspPage.java:2471)
     at cfmanage2dinventory2dedit2ecfm24489901.runPage(/var/www/vhosts/newride.ca/subdomains/admin/httpdocs/Assets/XHTML/manage-inventory-edit.cfm:98)
     at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
     at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:483)
     at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
     at cfindex2ecfm1182266435.runPage(/var/www/vhosts/newride.ca/subdomains/admin/httpdocs/index.cfm:27)
     at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
     at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:483)
     at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
     at cfApplication2ecfc1506803272$funcONREQUEST.runFunction(/var/www/vhosts/newride.ca/subdomains/admin/httpdocs/Application.cfc:126)
     at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:418)
     at coldfusion.runtime.UDFMethod$ArgumentCollectionFilter.invoke(UDFMethod.java:324)
     at coldfusion.filter.FunctionAccessFilter.invoke(FunctionAccessFilter.java:59)
     at coldfusion.runtime.UDFMethod.runFilterChain(UDFMethod.java:277)
     at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:192)
     at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:448)
     at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:308)
     at coldfusion.runtime.AppEventInvoker.invoke(AppEventInvoker.java:74)
     at coldfusion.runtime.AppEventInvoker.onRequest(AppEventInvoker.java:243)
     ... 27 more
Caused by: java.security.AccessControlException: access denied (java.io.FilePermission /opt/coldfusion8/runtime/servers/lib read)
     at java.security.AccessControlContext.checkPermission(AccessControlContext.java:323)
     at java.security.AccessController.checkPermission(AccessController.java:546)
     at java.lang.SecurityManager.checkPermission(SecurityManager.java:532)
     at java.lang.SecurityManager.checkRead(SecurityManager.java:871)
     at java.io.File.exists(File.java:731)
     at org.apache.bcel.util.ClassPath.getPathComponents(ClassPath.java:142)
     at org.apache.bcel.util.ClassPath.getClassPath(ClassPath.java:160)
     at org.apache.bcel.util.ClassPath.<init>(ClassPath.java:113)
     at org.apache.bcel.util.ClassPath.<clinit>(ClassPath.java:69)
     ... 70 more

     at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:70)
     at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
     at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
     at jrun.servlet.FilterChain.service(FilterChain.java:101)
     at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
     at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
     at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
     at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
     at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
     at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
     at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

Hi Sean,

What Java version?

CFadmin UI Server Settings > Settings Summary > JVM Details > Java Version

              

Likely there would be more details in the “coldfusion-out.log“ located in “/opt/coldfusion8/runtime/logs” that would probably show the reason.

I suspect the “500 ROOT CAUSE: coldfusion.runtime.EventHandlerException: Event handler exception” error commences once the CF has failed or hung.

HTH, Carl.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

I suspect the “500 ROOT CAUSE:  coldfusion.runtime.EventHandlerException: Event handler exception” error  commences once the CF has failed or hung.

Actually, no. I can browse most coldfusion sites & pages normally, but only certain specific pages cause the error [until I restart a couple of times] - interestingly enough, the java/jvm settings page is one of them!

Java version =                                      1.6.0_04 

-thanks

-sean

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

I see access denied in the thread there. Wonder if there has been a filesystem permission change of late or CF deamon user does not have full control (chmod ? sorry unix has been many years for me since).

Java 1.6.0_04 (comes with CF8 updater1) is fairly early. I gather prior to _10 there are some know leaks and performance issues. The current release is _22.


HTH again, Carl.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

I was just looking into that, it appears that I get slightly different behaviours running the init script as different users....

1. if I execute /opt/coldfusion8/bin/coldfusion start/restart/stop - AS root everything is generally pretty good [no option to start it as apache - not a real system user]

2. if I execute the etc/init.d/coldfusion_8 script and leave the RUNTIME_USER=""apache"" I usually get  a clean startup - but 'not always' the 500 error is still intermittent on restart but still always the same pages.

Looking at the permissions:

4.0K drwxr-xr-x 3 apache root 4.0K Jul 12  2009 coldfusion
4.0K drwxrwxr-x 2 apache root 4.0K Jul 12  2009 lib

hmmmm.....

I did change the initial heap & maximum heap to 256m - a suggestion from a tuning article, and the concurrent threads for pretty much everything WAY down and template/query caching WAY up - this seems to have solved the bulk of the startup errors [for now]  but the 500 error is still a bit of a mystery...

-sean

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

Permission look OK tho I wonder if I am suggesting the permission being an issue when it works OK sometimes and not another - there is a flaw in my reasoning there as since you have permission you should not loose the permission during the CF uptime.

Still keen to know what the OUT log says for the period when you get the 500 error (or is it the same as the browser error details you have provided)?

JVM memory would perhaps be better min 256  max 724 rather than 256 and 256 or nothing and 512.

HTH once more, Carl.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

I don't have an out log in that location ... I do have: /runtime/logs/coldfusion-event.log

if I search the whole system there is no out log [coldfusion 8.0.1 right?]

this is the tail end from a clean restart - somestimes it starts clean, sometimes not, I'm not convinced altering the java heap settings did anything.

11/15 19:53:18 user CFMxmlServlet: destroy
11/15 19:53:18 user CFCServlet: destroy
11/15 19:53:18 user CFFormGateway: destroy
11/15 19:53:18 user CFSwfServlet: destroy
11/15 19:53:18 user FlashGateway: destroy
11/15 19:53:18 user CFInternalServlet: destroy
11/15 19:53:30 info No JDBC data sources have been configured for this server (see jrun-resources.xml)
11/15 19:53:30 info JRun Proxy Server listening on *:51800
11/15 19:53:30 info Deploying enterprise application "Adobe_ColdFusion_8" from: file:/opt/coldfusion8/
11/15 19:53:30 info Deploying web application "Adobe ColdFusion 8" from: file:/opt/coldfusion8/
11/15 19:53:31 user JSPServlet: init
11/15 19:53:31 user ColdFusionStartUpServlet: init
11/15 19:53:31 user ColdFusionStartUpServlet: ColdFusion: Starting application services
11/15 19:53:31 user ColdFusionStartUpServlet: ColdFusion: VM version = 10.0-b19
11/15 19:53:35 user ColdFusionStartUpServlet: ColdFusion: application services are now available
11/15 19:53:35 user CFMxmlServlet: init
11/15 19:53:35 user CFMxmlServlet: Macromedia Flex Build: 87315.134646
11/15 19:53:36 user CFSwfServlet: init
11/15 19:53:36 user CFCServlet: init
11/15 19:53:36 user FlashGateway: init
11/15 19:53:36 user MessageBrokerServlet: init
11/15 19:53:37 user CFFormGateway: init
11/15 19:53:37 user CFInternalServlet: init
11/15 19:54:05 info No JDBC data sources have been configured for this server (see jrun-resources.xml)
11/15 19:54:05 info JRun Proxy Server listening on *:51800
11/15 19:54:05 info Deploying enterprise application "Adobe_ColdFusion_8" from: file:/opt/coldfusion8/
11/15 19:54:05 info Deploying web application "Adobe ColdFusion 8" from: file:/opt/coldfusion8/
11/15 19:54:06 user JSPServlet: init
11/15 19:54:06 error There is no web application configured to service your request
11/15 19:54:06 user NoWebappServlet: init
11/15 19:54:06 error There is no web application configured to service your request
11/15 19:54:06 user ColdFusionStartUpServlet: init
11/15 19:54:06 user ColdFusionStartUpServlet: ColdFusion: Starting application services
11/15 19:54:06 user ColdFusionStartUpServlet: ColdFusion: VM version = 10.0-b19
11/15 19:54:08 error There is no web application configured to service your request
11/15 19:54:10 user ColdFusionStartUpServlet: ColdFusion: application services are now available
11/15 19:54:10 user CFMxmlServlet: init
11/15 19:54:10 user CFMxmlServlet: Macromedia Flex Build: 87315.134646
11/15 19:54:11 user CFSwfServlet: init
11/15 19:54:11 user CFCServlet: init
11/15 19:54:11 user FlashGateway: init
11/15 19:54:11 user MessageBrokerServlet: init
11/15 19:54:12 user CFFormGateway: init
11/15 19:54:12 user CFInternalServlet: init

But I do get tons & tons of these:

Caused by: java.lang.NoSuchMethodException: jrunx.scheduler.SchedulerService.scheduleFromNow()

10 or so in a row every 10-15 seconds.

Here's what I just got from a non-clean start:

[root@impulsemedia logs]# tail -f ../runtime/logs/coldfusion-event.log
11/15 19:54:08 error There is no web application configured to service your request
11/15 19:54:10 user ColdFusionStartUpServlet: ColdFusion: application services are now available
11/15 19:54:10 user CFMxmlServlet: init
11/15 19:54:10 user CFMxmlServlet: Macromedia Flex Build: 87315.134646
11/15 19:54:11 user CFSwfServlet: init
11/15 19:54:11 user CFCServlet: init
11/15 19:54:11 user FlashGateway: init
11/15 19:54:11 user MessageBrokerServlet: init
11/15 19:54:12 user CFFormGateway: init
11/15 19:54:12 user CFInternalServlet: init
11/15 19:55:33 error SchedulerService not found
jrunx.kernel.ServiceException: SchedulerService not found
        at jrunx.kernel.ServiceAdapter.findObjectInstance(ServiceAdapter.java:638)
        at jrunx.kernel.ServiceAdapter.unscheduleRunnable(ServiceAdapter.java:764)
        at jrun.servlet.session.SessionService.unscheduleRunnable(SessionService.java:1019)
        at jrun.servlet.session.SessionHolder.reschedule(SessionHolder.java:148)
        at jrun.servlet.session.JRunSession.setMaxInactiveInterval(JRunSession.java:135)
        at jrun.servlet.session.SessionService.createSession(SessionService.java:1250)
        at jrun.servlet.ForwardRequest.getSession(ForwardRequest.java:384)
        at coldfusion.runtime.AppHelper.setupJ2eeSessionScope(AppHelper.java:543)
        at coldfusion.runtime.AppHelper.setupSessionScope(AppHelper.java:614)
        at coldfusion.tagext.lang.ApplicationTag.doStartTag(ApplicationTag.java:279)
        at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
        at cfApplication2ecfm1253482620._factor1(E:\cf8_updates\cfusion\wwwroot\CFIDE\administrator\Application.cfm:62)
        at cfApplication2ecfm1253482620._factor7(E:\cf8_updates\cfusion\wwwroot\CFIDE\administrator\Application.cfm:4)
        at cfApplication2ecfm1253482620.runPage(E:\cf8_updates\cfusion\wwwroot\CFIDE\administrator\Application.cfm:1)
        at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
        at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:483)
        at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
        at coldfusion.filter.CfincludeFilter.include(CfincludeFilter.java:33)
        at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:221)
        at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
        at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
        at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
        at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
        at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:74)
        at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)
        at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
        at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
        at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
        at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
        at coldfusion.CfmServlet.service(CfmServlet.java:198)
        at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
        at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
        at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42)
        at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
        at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
        at jrun.servlet.FilterChain.service(FilterChain.java:101)
        at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
        at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
        at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
        at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
        at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
        at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
        at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
        at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
        at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

11/15 19:55:33 error SchedulerService not found
jrunx.kernel.ServiceException: SchedulerService not found
        at jrunx.kernel.ServiceAdapter.findObjectInstance(ServiceAdapter.java:638)
        at jrunx.kernel.ServiceAdapter.unscheduleRunnable(ServiceAdapter.java:764)
        at jrun.servlet.session.SessionService.unscheduleRunnable(SessionService.java:1019)
        at jrun.servlet.session.SessionHolder.reschedule(SessionHolder.java:148)
        at jrun.servlet.session.JRunSession.setMaxInactiveInterval(JRunSession.java:135)
        at coldfusion.runtime.J2eeSessionScope.SetMaxInactiveInterval(J2eeSessionScope.java:99)
        at coldfusion.runtime.AppHelper.setupJ2eeSessionScope(AppHelper.java:559)
        at coldfusion.runtime.AppHelper.setupSessionScope(AppHelper.java:614)
        at coldfusion.tagext.lang.ApplicationTag.doStartTag(ApplicationTag.java:279)
        at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
        at cfApplication2ecfm1253482620._factor1(E:\cf8_updates\cfusion\wwwroot\CFIDE\administrator\Application.cfm:62)
        at cfApplication2ecfm1253482620._factor7(E:\cf8_updates\cfusion\wwwroot\CFIDE\administrator\Application.cfm:4)
        at cfApplication2ecfm1253482620.runPage(E:\cf8_updates\cfusion\wwwroot\CFIDE\administrator\Application.cfm:1)
        at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
        at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:483)
        at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
        at coldfusion.filter.CfincludeFilter.include(CfincludeFilter.java:33)
        at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:221)
        at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
        at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
        at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
        at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
        at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:74)
        at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)
        at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
        at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
        at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
        at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
        at coldfusion.CfmServlet.service(CfmServlet.java:198)
        at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
        at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
        at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42)
        at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
        at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
        at jrun.servlet.FilterChain.service(FilterChain.java:101)
        at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
        at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
        at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
        at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
        at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
        at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
        at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
        at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
        at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

11/15 19:59:29 user CFMxmlServlet: destroy
11/15 19:59:29 user CFCServlet: destroy
11/15 19:59:29 user CFFormGateway: destroy
11/15 19:59:29 user CFSwfServlet: destroy
11/15 19:59:29 user FlashGateway: destroy
11/15 19:59:29 user CFInternalServlet: destroy
11/15 19:59:41 info No JDBC data sources have been configured for this server (see jrun-resources.xml)
11/15 19:59:41 info JRun Proxy Server listening on *:51800
11/15 19:59:42 info Deploying enterprise application "Adobe_ColdFusion_8" from: file:/opt/coldfusion8/
11/15 19:59:42 info Deploying web application "Adobe ColdFusion 8" from: file:/opt/coldfusion8/
11/15 19:59:42 user JSPServlet: init
11/15 19:59:43 user ColdFusionStartUpServlet: init
11/15 19:59:43 user ColdFusionStartUpServlet: ColdFusion: Starting application services
11/15 19:59:43 user ColdFusionStartUpServlet: ColdFusion: VM version = 10.0-b19
11/15 19:59:47 user ColdFusionStartUpServlet: ColdFusion: application services are now available
11/15 19:59:47 user CFMxmlServlet: init
11/15 19:59:47 user CFMxmlServlet: Macromedia Flex Build: 87315.134646
11/15 19:59:47 user CFSwfServlet: init
11/15 19:59:47 user CFCServlet: init
11/15 19:59:48 user FlashGateway: init
11/15 19:59:48 user MessageBrokerServlet: init
11/15 19:59:48 user CFFormGateway: init
11/15 19:59:48 user CFInternalServlet: init

-arrggh.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

hmm, this is the only entry I have in the scheduler.log

"Severity","ThreadID","Date","Time","Application","Message"
"Information","jrpp-4","09/22/10","13:19:18",,"/opt/coldfusion8/logs/scheduler.log initialized"

hmmm 3600+ errors going back to roughly that date...  curiously the same date as the mysterious resource poroblems started showing up.

hm.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

what is this???

I get the errors if I log into cfadministrator and go to the java/jvm page ...  and here is an odd line:

at cfruntime2ecfc1355728568$funcGETPATH.runFunction(E:\cf8_updates\cfusion\wwwroot\CFIDE\adminapi\runtime.cfc:1775)

?????? that path can't exist on a linux box?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

Let's have a look at the JVM.CONFIG \opt\coldfusion8\runtime\bin

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

there appear to be 2 of them, identical, /opt/coldfusion8/runtime/bin/jvm.config & /opt/coldfusion8/bin/jvm.config

# cat /opt/coldfusion8/runtime/bin/jvm.config
#
# VM configuration
#
# Where to find JVM, if {java.home}/jre exists then that JVM is used
# if not then it must be the path to the JRE itself
java.home=/opt/coldfusion8/runtime/jre
#
# If no java.home is specified a VM is located by looking in these places in this
# order:
#
#  1) bin directory for java.dll (windows) or lib/<ARCH>/libjava.so (unix)
#  2) ../jre
#  3) registry (windows only)
#  4) JAVA_HOME env var plus jre (ie $JAVA_HOME/jre)
#

# Arguments to VM
java.args=-server  -Djava.awt.headless=true -Djava.security.manager "-Djava.security.policy=/opt/coldfusion8/lib/coldfusion.policy" "-Djava.security.auth.policy=/opt/coldfusion8/lib/neo_jaas.policy" -Xms256m -Xmx512m -Dsun.io.useCanonCaches=false -XX:MaxPermSize=192m -XX:+UseParallelGC -Dcoldfusion.rootDir={application.home}/../ -Dcoldfusion.libPath={application.home}/../lib -Dcoldfusion.classPath={application.home}/../lib/updates,{application.home}/../lib,{application.home}/../gateway/lib/,{application.home}/../wwwroot/WEB-INF/flex/jars,{application.home}/../wwwroot/WEB-INF/cfform/jars

#
# commas will be converted to platform specific separator and the result will be passed
# as -Djava.ext.dirs= to the VM
java.ext.dirs={jre.home}/lib/ext

#
# where to find shared libraries
java.library.path={application.home}/../lib
system.path.first=false

#
# set the current working directory - useful for Windows to control
# the default search path used when loading DLLs since it comes
# before system directory, windows directory and PATH
java.user.dir={application.home}/../../lib

# JVM classpath
java.class.path={application.home}/servers/lib,{application.home}/../lib/macromedia_drivers.jar,{application.home}/lib/cfmx_mbean.jar,{application.home}/lib

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

>there appear to be 2 of them,
Maybe one is a link to other

>JVM.CONFIG

Well there are no strange Windows e:\ paths in there. There are some odd spaces however guess this is just a forum pasting problem "/cffor m/jars

../l ib/macromedia  jar,{appli cation". Some of the -Djava.security... are new to me tho since alarm bells have not gone off I suspect are normal for the envoronment.

Sorry Sean not very helpful, Carl.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

absolutely - one is a symlink

0 lrwxrwxrwx 1 root root 39 Jul 12  2009 /opt/coldfusion8/bin/jvm.config -> /opt/coldfusion8/runtime/bin/jvm.config

starting to run out of ideas here ....  if I try to hit the init script with a status request #service coldfusion_8 status - I get the java out of memory error/dump ..... dunno...

so far I have:

- 500 errors on some pages - same pages all the time, I can browse the rest cfm pages properly

- the 500 error only appears ~sometimes~ on a given restart, but the error remains consistent if it appears.

- when cf8 ~appears~ to hit it memory limit httpd processes seem to hang even though apache should be killing them

kinda stuck

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

ok - now it's getting WEIRD .... I decided to see what the similarities were for the pages tht consistently got the 500 errors on a "bad" cf restart....  I copied a fairly simple one to a different domain with no Application.cfc and tried to execute the file - not surprisingly I got the 500 error, ok - I would have expected something more serious as it had object dependancies ... no matter....

ok - time to find the specific line.

I ended up with a completely empty file - no cf code, no html....  guess what? I still got the 500 error.

ok !CACHE! - no, I emptied the coldfusion template cache still got the error, retried it & restarted ... still got the error.

WTF?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

I got nothing more to add.  Hopeful someone else in the CF community will pick it up and provide input. I for one am interested to see the solution.

Best Regards, Carl.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

hmmmm:

seems maybe this whole thing came down to some saved class files?

/opt/coldfusion8/wwwroot/WEB-INF/cfclasses seems to contain all the compiled java classes & is persistent from one restart to another.

the page in question that I copied from one site to another had a class file here,

If i toggled the " " setting I could get the pages to display without the 500 error toggled off, as soon as I toggled it back on - I get the 500 error again.

I delete the class file in question  "cfbuy2dmy2dcar2ecfm1758598524.class" restart cf - visit the offending page with save class files on expecting an error, I got the page, no error & a new  cfbuy2dmy2dcar2ecfm1758598524.class  file was generated.   now with the save class file setting on OR off... everything is looking better & I am not getting the error on any pages [even the ones in the cfadmin]

SO:

It's looking pretty much like a corrupted file for the bulk of the problems [I think]

- what in the hell is /opt/coldfusion8/wwwroot/WEB-INF/cfclasses anyway? is that the memory cache just compiled & saved to disk???

- can I safely delete all the class files in there when something goes to hell?

I dunno - we'll see.

On the bright side, looks like I can get away with about half the RAM I had allocated to the server in the first place!

Thanks all for your help & suggestions.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Nov 15, 2010 Nov 15, 2010

Copy link to clipboard

Copied

Just getting to this thread. Interesting things, for sure. I have a few more thoughts to add over Carl's admirable effort to jump in while you were in panic mode.

First, as for your last note below, I do find it odd that deleting the class file made things work. As for the cfclasses directory, it's the place where classes are "saved" if you enable that option "save class files". If you don't use that, then each time a page is requested and it's NOT found in the template cache, it's loaded from disk instead. As you noticed, these saved class files are saved over restarts, which would usually be a good thing. I've never heard of a situation where deleting a class file solves such a dramatic problem as you reported. I'd really wonder if there may not be more going on.

Second, along those lines of finding out what more may be going on, you guys were talking earlier about where to find additional log info. Carl offered the \runtime\logs, and indeed they can offer valuable additional diagnostic data, but you couldn't find these "-out.logs" because on Linux instead that data is written to the \logs\cfserver.log.

Third, as for the strange path you'd seen in the errors (in your case, E:\cf8_updates\cfusion\wwwroot\CFIDE\adminapi\runtime.cfc), these are not referring to files that really exist on your environment. Instead, these are CFML files that came with CF and were precompiled (so you can’t see the source). CF stores internally their path where they were when precompiled (on the box of the Adobe CF engineer). Just ignore the location.

Fourth, you also mentioned seeing errors related to the scheduler, but these have NOTHING to do with scheduled tasks. It's an understandable confusion. Instead, these refer to internal threads that the CF//JRun engine runs to do various background processes. Really, you need not likely worry about them at all. Instead, the errors you're seeing are likely long after the real root cause problem. Again, the cfserver.log should help you find that, but the key is not to look only at what was happening right before a crash (or slowdown) but instead find the last sign of no errors in the log prior to that (which may be seconds or minutes previous) and see what errors start happening at that point.

Finally, you may also find useful information in some unexpected (and often missed) logs, which you may find in the /Runtime/bin directory. These (if there) are logs that are made when the JVM crashes, and they (though not very big) may offer insight into what's happening. For instance, you could be crashing for any of several outofmemory conditions, some of which would mean you should increase your heap (your offered size of 256 is awfully small), but some outofmemory messages may indicate a problem having nothing to do with the heap size.

Let us know if these thoughts or your own investigations find things.

/charlie arehart

charlie@carehart.org

Providing CF and CFBuilder troubleshooting services

at http://www.carehart.org/consulting


/Charlie (troubleshooter, carehart.org)

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Nov 16, 2010 Nov 16, 2010

Copy link to clipboard

Copied

Thanks Dave:

You can safely disregard that file path. Precompiled templates contain the path from which they were originally compiled.

- noted.

Charlie;

Very comprehensive, thank you, and yes a nod to Carl is due as well.[Thanks again Carl]

I think I may have said "panic mode" maybe I should downgrade that to severe frustration mode - at least mail, dns & most httpd were unaffected & I did get a blast email out noting "emergency maintenance" so any damage was/is minimized.....

on to it....

I've never heard of a situation where deleting a class file solves such a  dramatic problem as you reported. I'd really wonder if there may not be  more going on.

Me too, the closest I can figure is that the class files may be related to one another in some way [like a database] i.e. one function called on page A is compiled, the CF comes across the same function on pages B, C & D.... instead of compiling that one function into classes for B, C & D icompiles classes for those pages WITHOUT the one function & just a pointer or reference to class file A.

Disclaimer: That was a WAG [Wild Wssed Guess] - I have no knowledge of how the class files are compiled or related to one another - but it would explain why deleting the one offending file made the rest of the classes work again.

Logs:

yea- I pretty much figured they were platform differences, in fact you can convince CF to log everything to syslog facilities with a little acrobatics... but worthwhile?? prolly not.

Strange Path:

Dave mentioned as wel... noted, thanks.

Scheduler/Logs/When to look:

Yes, the scheduler thing became apparent when I tested it - worked - I encourage users to use cron instead.... so there were no tasks scheduled, anyway... as far as logging goes and how far back, it's very difficult as this seems to be an issue that pops up 'whenever' I might go for 2 weeks or 2 months without an issue [actually I ran for close to a year with no problems before July this year] by the time I'm looking at logs the problem has already been rotated out and pruned [yea- yea - I know, increase log file size & number of archives , time to live ] Also looking at those java logs sucks, I can find my way around any linux syslem log easily, but that java stuff makes my eyes bleed!

cf/Runtime/bin:

Ahhhhhhh! these are very interesting indeed, however the last if from Sept 25....  a little too far back, but deservers some investigation anyway.

Memory heep too low, yes - I am trying to make it crash so I can examine/dfferentiate the errors...  [though so far it is running quite smooth and smart - a lot of the sites on the server are not actual loaded applications, usually simple website includes, light scripting and maybe a few database calls - so the load on CF is not heavy at all]

The one thing I did notice [or rather jumped out at me] was that the java log in cf/Runtime/bin reported 8-4core CPUs available to it.

Keeping in mind that this is a VM and those CPU's are shared, I have had significant problems with php caching accelerators Xcache to be specific when I tried to use more than 1 CPU. I'm curious if my concurrent threads in CFAdmin could have been causing problems....  according to the official docs, I should be able to set my threads to between 8 & 16... [is it number of CPUs = threads x 2 or number of CORES? i.e. 32 - 64 threads?????] 

In the past if I tried to set xcache to more than 2 CPU, I got an immediate & unforgiving ticket to hell while being escorted by Nazis. Apache would not even start.

Still testing & monitoring.

-thanks a bunch [everyone!]

-sean

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Nov 16, 2010 Nov 16, 2010

Copy link to clipboard

Copied

Hi Sean,

Glad to jump in and help out  where I can (if only to be someone to bounce idea's off / sounding board).

I was thinking delete cfclasses - but been flamed for suggesting such before so held back.

Anyway how has it been running?

Cheers, Carl.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Nov 17, 2010 Nov 17, 2010

Copy link to clipboard

Copied

no errors or memory problems [yet?]

seems nice and stable now.....

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Nov 17, 2010 Nov 17, 2010

Copy link to clipboard

Copied

Happy Days.

Being interested I wonder if the OUT logs which Charlie pointed mentioned are cf\logs\cfserver.log would provide clue as to problem. Are you able to look back thru the log to when problems where happening and see if there are any salient details? If there is something noteworthy perhaps you can paste a  sample?

Cheers, Carl.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Nov 17, 2010 Nov 17, 2010

Copy link to clipboard

Copied

hmmm interesting - there are nothing but a few scheduler errors in the cfserver log since the 15th ....  guess that's a good thing.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Resources
Documentation