The CF11 release installer laid down Java 7. The latter CF11 installer uses Java 8. If one went release then U1 to U5 that would not alter the underlying Java version.
I have seen issues where database connections are broken due to a poorly performing Java heap or non heap issue. You could do well to check how well the heap or non heap IE metaspace values are performing and if one or other need adjustment. Not to be forgotten with Java memory is CodeCache and CompressedClass to know either of those are consuming memory in good boundaries. Use traditional Java logging to a file or enable JMX then use JDK tools like jconsole or jmc to check the memory consumption is stressed or not.
I noticed our development server still had -XX:MaxPermSize=192m where the new production server had -XX:MaxMetaspaceSize=192m set instead. Which reading up on that setting not having it defined would have given an unlimited access size. I've changed the development server to limit to 192m to see if it causes issues there. Are there any good articles written on metaspace tuning for CF like the connector one?
We don't run the internal Java but instead external JRE copies so both machines are running JRE 1.8.0_45 (working on upgrading to _51 soon).
Regarding the Performance issue, here are some resources, that will help you.
* Performance tuning for ColdFusion applications (http://www.adobe.com/devnet/coldfusion/articles/coldfusion_performance.html),
These articles are for JRE 7, however, the logic remains the same for JRE 8 as well.
After setting my development system to have the same -XX:MaxMetaspaceSize=192m I started getting the same errors just browsing around a development website.
Error Executing Database Query.
Timed out trying to establish connection
java.sql.SQLException: Timed out trying to establish connection
Seems to me you are creeping up on the answer since setting dev to -XX:MaxMetaspaceSize=192m
you get the database connectivity problem. So you are consuming greater than 192m of MetaSpace which
is not unusual on 64bit.
With Java 8 it is able to manage MetaSpace automatically if you do not define a value for MaxMetaspaceSize= however it may not make good decisions and will be inclined to re-size down to it's lower value. All this resizing invokes a Full garbage collection which is a performance loss.
You can do well to define an initial and maximum setting for MetaSpace. I find this works well in most cases:
Having said that some Java logging or JMX monitoring might reveal better values to apply given the
type of workload.
HTH again, Carl.
I enabled JMX and installed VisualVM so I've been able to watch the resource usage. Oddly the DB timeout happened when only 80mb of metaspace was in use when limited to 192. We were doing similar min/max settings with the previous Java 7 settings, thank you for providing me the terms for the new settings.
I'm also looking into tomcat connector tuning as well but I don't think the DB timeouts are related to it (at least I don't think they are).
It is a good idea to apply tomcat tuning tho like you say adjusting tomcat pools, threads and timeout is probably not related to the database timeout problem.
I guess you have adjusted MetaSpace. Does Jvisualvm show it consuming more memory now?
While you have JMX enabled you might like use jconsole or jmc to check CodeCache and CompressClass memory consumption to know those values are well.
Unfortunately the increased memory space and Tomcat tuning did not resolve the issue on our production servers. I was hoping they might happen less frequently but they still seem to happen at about the same rate.
I've created a Adobe Bug for the timeouts as we've been unable to make them stop on our production environment and can reproduce them on development as well.
After countless attempts to stop this from happening the solution we finally found was to run the Microsoft JDBC drivers v4.2 inside of ColdFusion11. Once we stopped using all Adobe JDBC drivers to connect to our MS-SQL 2014 server the server stopped being unstable. If even one active DB is using the built in Adobe MS-SQL driver then the system in both our development and production enviroment will eventually behave as documented previously.
- CF11 CFIDE -> Data Sources -> Type name as usual but choose “other” as driver
- JDBC URL is where you define the connection string including the server to connect to, the DB, and encryption (if you use encryption make sure to use the FQDN
- Driver Class must contain the name of the JDBC driver
- Driver Name must contain the correct name
- Fill out the User Name and Password as usual
- The Advanced area contains all of the same settings except for the “Max Pooled Statements”
I have a set of benchmark scripts that run an exhaustive set of DB queries I use to evaluate performance changes over time. On my development test server these take 630 seconds to run on average and moving to the MS JDBC driver the on average time dropped by 30 seconds to complete for unencrypted connections. For encrypted connections it took on average the same amount of time to complete.
One issue we’ve discovered is that the driver enforces some MS coding practices by disallowing the running of certain queries. For example a select identity inside an insert query will error using the MS JDBC driver but run with the Adobe driver.