We seem to have run in to a problem with two CF instances having high CPU usage after 1-2 hours.
We have three servers, two of which are back ends to a load balancer, the third is just on its own. They all have the default installation of CF 11 Enterprise running on W2012 R2 / IIS 8
I noticed that CF installed on the two backend servers started using a lot of CPU (30-40%) after a couple of hours, then as time went on further so did the CPU usage. If left overnight it would be hitting 100% constantly.
These are fresh installs and have no content being served except for the default web site in IIS. So this rules out dodgy code etc.
All three servers have the same software installed on them and the same IIS configuration. I went through the CF 11 lockdown guide to see if this would make any difference, the issue occurs with or without the changes in this guide. I have changed the jre to a different version and this has made no difference. I have made changes recommended for the workers.properties files, still the same.
The only difference between these two servers and the third server is that the two are behind a load balancer. This LB polls the servers to see if it will get a response. I enabled the firewall on one of the servers to block all incoming traffic and the problem has not occurred since. So I assume it has something to do with the request being sent from the load balancers?
They send OPTIONS request every 3 seconds to both of these servers that have the problem. Could this be causing a problem with CF 11??
I have looked at the threads through ProcExp when the problem occurs and can see the threads related to msvcr100.dll seem to be the ones that are using up all the CPU. This file is the same through out all our servers and even after reinstalls etc the problem still occurs, so I don't think this file is corrupt.
Any one got any other ideas?
C:\ColdFusion11\cfusion\bin has a msvcr100.dll. Perhaps after couple hours of uptime one of the Java memory spaces has filled with used objects and Java is in a state of performing full garbage collections however not able to release any memory. Perhaps set a larger value for Java Generations in:
CF11admin > Server Settings > Java and JVM >
Minimum JVM Heap Size (MB) say 1024
Maximum JVM Heap Size (MB) say 3072
and JVM Arguments > XX:MaxPermSize=512m
Values mentioned assuming your server has 4 - 8Gb RAM.
I did increase the heap size to min 1024, max 4026 but not the perm size. This didn't seem to make a difference. I will do the perm size as well to see if this helps.
I stopped all traffic to one of the servers from the load balancer last night and the CPU has stayed normal, so it seems to have some tie in with the traffic it sends.