Coldfusion Service keeps stopping with these two errors Event 5002 or 5009

[trenab83858724]

I have five (5) applications that are using Coldfusion 10.  Each application is using the same Default Pool because I don't know how to separate the Default Pools. How do I get the service to stop going down?

[haxtbh]

Have a look at the coldfusion-out.log/coldfusion-error.log file in the cfusion/logs folder. This should give you more of a clue as to what the services is doing before its shuts down

[trenab83858724]

Okay. This is from the coldfusion-out.log but, I don't know what it means. 

Aug 1, 2016 08:00:00 AM Information [DefaultQuartzScheduler_Worker-10] - Task DEFAULT.CFUPDATER - AUTO UPDATES triggered.

Aug 1, 2016 08:00:00 AM Information [DefaultQuartzScheduler_Worker-10] - Starting HTTP request {URL='http://localhost:80/CFIDE/administrator/cfupdater/Update.cfc?method=autoInternalUpdate&RequestTimeou...', method='get'}

Aug 1, 2016 08:00:00 AM Information [DefaultQuartzScheduler_Worker-10] - HTTP request completed  {Status Code=404 ,Time taken=15 ms}

Aug 1, 2016 08:21:35 AM Information [scheduler-0] - Run Client Storage Purge

Aug 1, 2016 08:49:17 AM Information [pool-2-thread-2] - Stopping Event Gateways.

Aug 1, 2016 08:49:22 AM Information [pool-2-thread-2] - ColdFusion stopped

[haxtbh]

What about the coldfusion-error.log.

Is that all of the coldfusion-out log?

What update level do you have on these CF10 servers?

[trenab83858724]

That is not all that is in the log.  I just chose a portion where the shutdown took place.

Here is the error log for Aug. 1st:

Aug 01, 2016 8:50:20 AM org.apache.catalina.core.AprLifecycleListener init

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\\ColdFusion10\\cfusion\lib;C:\\ColdFusion10\\cfusion\jintegra\bin;C:\\ColdFusion10\\cfusion\jintegra\bin\international;C:\\ColdFusion10\\cfusion\lib\oosdk\classes\win

Aug 01, 2016 8:50:23 AM org.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler ["ajp-bio-8012"]

Aug 01, 2016 8:50:23 AM org.apache.catalina.core.StandardService startInternal

INFO: Starting service Catalina

Aug 01, 2016 8:50:23 AM org.apache.catalina.core.StandardEngine startInternal

INFO: Starting Servlet Engine: Apache Tomcat/7.0.23

Aug 01, 2016 8:50:25 AM org.apache.catalina.util.SessionIdGenerator createSecureRandom

INFO: Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [1,047] milliseconds.

Aug 01, 2016 8:48:28 AM org.apache.catalina.core.ApplicationContext log

INFO: ColdFusionStartUpServlet: ColdFusion: Starting application services

Aug 01, 2016 8:48:28 AM org.apache.catalina.core.ApplicationContext log

INFO: ColdFusionStartUpServlet: ColdFusion: VM version = 23.7-b01

Aug 01, 2016 8:48:35 AM org.apache.catalina.core.ApplicationContext log

INFO: CFMxmlServlet: Macromedia Flex Build: 87315.134646

Aug 01, 2016 8:48:43 AM org.apache.coyote.AbstractProtocol start

INFO: Starting ProtocolHandler ["ajp-bio-8012"]

Aug 01, 2016 8:48:43 AM com.adobe.coldfusion.launcher.Launcher run

INFO: Server startup in 33587 ms

Aug 01, 2016 8:58:46 AM org.apache.catalina.core.AprLifecycleListener init

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\\ColdFusion10\\cfusion\lib;C:\\ColdFusion10\\cfusion\jintegra\bin;C:\\ColdFusion10\\cfusion\jintegra\bin\international;C:\\ColdFusion10\\cfusion\lib\oosdk\classes\win

Aug 01, 2016 8:58:48 AM org.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler ["ajp-bio-8012"]

Aug 01, 2016 8:58:49 AM org.apache.catalina.core.StandardService startInternal

INFO: Starting service Catalina

Aug 01, 2016 8:58:49 AM org.apache.catalina.core.StandardEngine startInternal

INFO: Starting Servlet Engine: Apache Tomcat/7.0.23

Aug 01, 2016 8:58:50 AM org.apache.catalina.util.SessionIdGenerator createSecureRandom

INFO: Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [859] milliseconds.

Aug 01, 2016 8:59:05 AM org.apache.catalina.core.ApplicationContext log

INFO: ColdFusionStartUpServlet: ColdFusion: Starting application services

Aug 01, 2016 8:59:05 AM org.apache.catalina.core.ApplicationContext log

INFO: ColdFusionStartUpServlet: ColdFusion: VM version = 23.7-b01

Aug 01, 2016 8:59:12 AM org.apache.catalina.core.ApplicationContext log

INFO: CFMxmlServlet: Macromedia Flex Build: 87315.134646

Aug 01, 2016 8:59:20 AM org.apache.coyote.AbstractProtocol start

INFO: Starting ProtocolHandler ["ajp-bio-8012"]

Aug 01, 2016 8:59:20 AM com.adobe.coldfusion.launcher.Launcher run

INFO: Server startup in 36785 ms

Aug 01, 2016 9:12:19 AM org.apache.catalina.core.AprLifecycleListener init

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\\ColdFusion10\\cfusion\lib;C:\\ColdFusion10\\cfusion\jintegra\bin;C:\\ColdFusion10\\cfusion\jintegra\bin\international;C:\\ColdFusion10\\cfusion\lib\oosdk\classes\win

Aug 01, 2016 9:12:21 AM org.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler ["ajp-bio-8012"]

Aug 01, 2016 9:12:22 AM org.apache.catalina.core.StandardService startInternal

INFO: Starting service Catalina

Aug 01, 2016 9:12:22 AM org.apache.catalina.core.StandardEngine startInternal

INFO: Starting Servlet Engine: Apache Tomcat/7.0.23

Aug 01, 2016 9:12:23 AM org.apache.catalina.util.SessionIdGenerator createSecureRandom

INFO: Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [719] milliseconds.

Aug 01, 2016 9:12:37 AM org.apache.catalina.core.ApplicationContext log

INFO: ColdFusionStartUpServlet: ColdFusion: Starting application services

Aug 01, 2016 9:12:37 AM org.apache.catalina.core.ApplicationContext log

INFO: ColdFusionStartUpServlet: ColdFusion: VM version = 23.7-b01

Aug 01, 2016 9:12:45 AM org.apache.catalina.core.ApplicationContext log

INFO: CFMxmlServlet: Macromedia Flex Build: 87315.134646

Aug 01, 2016 9:12:52 AM org.apache.coyote.AbstractProtocol start

INFO: Starting ProtocolHandler ["ajp-bio-8012"]

Aug 01, 2016 9:12:54 AM com.adobe.coldfusion.launcher.Launcher run

INFO: Server startup in 37405 ms

Aug 01, 2016 9:31:46 AM org.apache.catalina.core.AprLifecycleListener init

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\\ColdFusion10\\cfusion\lib;C:\\ColdFusion10\\cfusion\jintegra\bin;C:\\ColdFusion10\\cfusion\jintegra\bin\international;C:\\ColdFusion10\\cfusion\lib\oosdk\classes\win

Aug 01, 2016 9:31:49 AM org.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler ["ajp-bio-8012"]

Aug 01, 2016 9:31:49 AM org.apache.catalina.core.StandardService startInternal

INFO: Starting service Catalina

Aug 01, 2016 9:31:49 AM org.apache.catalina.core.StandardEngine startInternal

INFO: Starting Servlet Engine: Apache Tomcat/7.0.23

Aug 01, 2016 9:31:51 AM org.apache.catalina.util.SessionIdGenerator createSecureRandom

INFO: Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [859] milliseconds.

Aug 01, 2016 9:32:03 AM org.apache.catalina.core.ApplicationContext log

INFO: ColdFusionStartUpServlet: ColdFusion: Starting application services

Aug 01, 2016 9:32:03 AM org.apache.catalina.core.ApplicationContext log

INFO: ColdFusionStartUpServlet: ColdFusion: VM version = 23.7-b01

Aug 01, 2016 9:32:10 AM org.apache.catalina.core.ApplicationContext log

INFO: CFMxmlServlet: Macromedia Flex Build: 87315.134646

Aug 01, 2016 9:32:17 AM org.apache.coyote.AbstractProtocol start

INFO: Starting ProtocolHandler ["ajp-bio-8012"]

Aug 01, 2016 9:32:17 AM com.adobe.coldfusion.launcher.Launcher run

INFO: Server startup in 35100 ms

I cannot make heads or tails of the error log.

I don't know what the update level is?  Where do I find that?

[haxtbh]

If you log into the coldfusion admin and click the Settings Summary from the menu.

Copy and paste the full version number.

What you posted is just the logs for coldfusion starting up. It looks like you are still just showing portions. Really need the whole log.

Did you find the coldfusion-error.log?

[carl type3]

Hi,

Not to be forgotten with CF10 update level is if you have performed the manual step WSCONFIG to increment the tomcat webserver connector to match CF10 update?

Perhaps some tomcat tuning is required?

Here are some notes on tomcat tuning from a talk I did a few years back. Many folks have commented that connection_pool_timeout and connectionTimeout setting have helped their performance.

Note "properties" changes require webserver (IIS perhaps) restart. Server.xml changes require CF10 restart.

Tomcat – Pool

● connection_pool_size

The number of connections made to the AJP backend that are maintained as a

connection pool. Tomcat strongly recommend adjusting this value for IIS to the

number of requests one web server process should be able to send to a backend in

parallel. You should measure how many connections you need during peak activity

without performance problems, and then add some percentage depending on your

growth rate. For IIS the default value is 250.

● connection_pool_minsize

Minimum size of the connection pool that will be maintained. Its default value is

(connection_pool_size+1)/2.

● connection_pool_timeout

Each thread could open an ajp13 connection if it has to forward a request to Tomcat,

creating a new ajp13 thread on Tomcat side. Keep this time interval in sync with the

connectionTimeout attribute of your AJP connector in Tomcat's server.xml. Note the

value for connector is given in seconds, the one in server.xml use milliseconds.

Default value 0

refer http://tomcat.apache.org/connectors-doc/reference/workers.html

Tomcat – workers.properties

● ColdFusion10\config\wsconfig\N\

worker.list=cfusion

worker.cfusion.type=ajp13

worker.cfusion.host=localhost

worker.cfusion.port=8012

worker.cfusion.max_reuse_connections=250

worker.cfusion.connection_pool_size = n00

worker.cfusion.connection_pool_minsize=size+1 / 2

worker.cfusion.connection_pool_timeout = n0

Tomcat - Threads

● maxThreads

The maximum number of request processing threads to be created by

this Connector, which therefore determines the maximum number of

simultaneous requests that can be handled. If not specified, this

attribute is set to 200.

● minSpareThreads

The number of request processing threads that will be created when

this Connector is first started. The connector will also make sure it

has the specified number of idle processing threads available. This

attribute should be set to a value smaller than that set for

maxThreads. The default value is 10.

● ConnectionTimeout

The number of milliseconds this Connector will wait, after accepting a

connection, for the request URI line to be presented. The default

value for AJP protocol connectors is -1 (i.e. infinite)

refer http://tomcat.apache.org/tomcat-7.0-doc/config/ajp.html

tomcat server.xml AJP13

● \ColdFusion10\cfusion\runtime\conf

<!-- Define an AJP 1.3 Connector on port 8009

-->

<Connector port="8012" protocol="AJP/1.3"

redirectPort="8445"

tomcatAuthentication="false"

maxThreads="n00"

minSpareThreads="max+1 / 2"

connectionTimeout="n0000" />

Where n = number.

HTH, Carl.

[trenab83858724]

Hi Carl,

I made the changes you suggested and now one of the users is having an issue with downloading their Excel reports.  Could this be the cause of the problem?

Thanks.

[carl type3]

Hi,

Did the 5002 or 5009 events cease?

I think the user excel download problem is a whole other matter. Suggest respond to this thread if original problem is solved or not. Create a new thread about the excel issue with some diagnostic about what the excel issue is and how it relates to the backend CF.

Regards, Carl.

[trenab83858724]

I thought it was probably another issue but, I'm still having an issue with the 5002 and 5009.  It's not happening as often but, it's still a problem.

[carl type3]

Hi,

Please note there were some Microsoft July security updates to desktops win 7,8 and 10 that effected excel downloads. If the resultant excel sheet is a light grey colour (color) then it could be related to the way excel download happens and recent MS update.

Back to 5002 5009.

- What update level did you find CF10 is at?

- Was WSCONFIG run to match tomcat IIS connector with CF tomcat level that some updates increment?

- What values did you use for "n" in workers.properties and server.xml tomcat configuration files?

Not discussed yet but could relate. What Java version is CF10 using? Have you done any CF Java and JVM changes?

Regards, Carl.

[trenab83858724]

I am still having this issue.  I agree that it has something to do with the IIS Application Pool Availability.  I have 5 (five) applications sharing the same Default Pool.  These are Cold Fusion 10 applications but, I don't know how to separate the Default Pools to allow each application to have it's own default pool.  Does anyone have any suggestions?

[carl type3]

Hi,

This official CF blog entry and the comments within might be helpful:

ColdFusion 11 IIS Connector Tuning » Adobe ColdFusion Blog

Regards, Carl/

[Carl Von Stetten]

Are your 5 applications running as separate "sites" in IIS, or through one main "site"?

[trenab83858724]

Yes.  They are separate sites.

[Carl Von Stetten]

Each site can be configured to use a different IIS App Pool.  There are some helpful suggestions on how to do this in the ColdFusion Lockdown Guide.  Basically, click on the Application Pools heading in the left pane of IIS Manager.  Create a pool for each site.  You should be able to set the .NET CLR version to "No Managed Code", unless you do .NET stuff too.

Then go to each site, click on Basic Settings... in the right pane of IIS Manager.  Click the Select... button on the dialog to choose an application pool.  Then click OK buttons to finish.  You might need to restart each site after switching the app pool.

HTH,

-Carl V.

[BKBK]

I think those are Event IDs for errors resulting from IIS Application Pool Availability. Microsoft gives you suggestions on how to debug or solve Event 5002 and Event 5009.