CF8 Performance Issues

[MikeP]

Everyone has been raving about how much faster CF8 is than previous versions. This got me very excited at first. Well, I've been running CF8 on my development machine for a couple weeks now. It's been painfully slow compared to CF7, but I've been putting up with it thinking that I must be missing some crucial setting that makes it fast. Well, I've checked every setting, tried every tweak, uninstalled/reinstalled, you name it - but it was still slow.

So today I decided to run my own crude performance test. I turned off all other software on my machine. I wrote a quick and dirty app, borrowing some code from the app used in the CF8 Performance Whitepaper. I adjusted the number of iterations in the loops so each test area would take about 2000 milliseconds on CF8.

Instead of measuring the times externally using a program like JMeter, I did a getTickCount() style measurement around the individual tests and outputted the results to the screen. To get the actual times I used a custom timer class I wrote in Java that uses JNI to get a far more accurate (sub-millisecond) timing. If you're tempted to blame my timer, please remember that it was slow performance in general that drove me to this. CF's built-in timing mechanisms reflect the same slowness. I just used my own timer so I could get a more accurate picture.

Both scenarios used the following server config.
- Blank application.cfm
- Trusted cache on
- Debug off
- Robust exceptions off
- All profilers and memory monitors, etc in CF8 were turned off.
- All non-essential external programs were closed.
- Verity Search service was disabled.
- 512MB minimum/maximum JVM heap size.

I ran ran the test cycle 6 times on both versions of CF. Each test cycle ran three blank tests (inclusion of a blank template) at the start and end as a "control group." Below I took my best CF8 results and compared them with my worst CF7 results.

-------------------------
Best CF8 Times:
-------------------------
blank: 0.157 ms
blank: 0.136 ms
blank: 0.141 ms
cfparam: 2,113.305 ms
component: 2,137.075 ms
dateFunctions: 1,939.005 ms
evaluate: 1,791.225 ms
list: 1,849.384 ms
REFindAndNoCase: 1,937.942 ms
REReplaceAndNoCase: 2,609.426 ms
blank: 0.149 ms
blank: 0.138 ms
blank: 0.134 ms

Total Time: 14,378.217 ms

--------------------------
Worst CF7 Times:
--------------------------
blank: 0.081 ms
blank: 0.071 ms
blank: 0.070 ms
cfparam: 301.511 ms
component: 1,369.429 ms
dateFunctions: 197.770 ms
evaluate: 237.860 ms
list: 68.813 ms
REFindAndNoCase: 75.692 ms
REReplaceAndNoCase: 141.153 ms
blank: 0.043 ms
blank: 0.030 ms
blank: 0.030 ms

Total Time: 2,392.552 ms

So for me CF7 was about 600% faster than CF8. I'm now back on CF7 and everything is fast again, but I really want to use CF8. I'm looking for suggestions as to why it could have been so slow for me. Have there been any common configuration issues that have caused performance bottlenecks? I have tried all of the obvious stuff. Turning on trusted cache, disabling debugging, giving more memory to the JVM heap, reinstalling, etc. Nothing seems to work. Next I'll probably try it on another machine, but I'm hoping for some suggestions in the mean time.

[Newsgroup_User]

In article <fcbqbe$62k$1@forums.macromedia.com> "MikeP"
<webforumsuser@macromedia.com> wrote:
> Everyone has been raving about how much faster CF8 is than previous
> versions. [big snip]

Weird. I'm consistently seeing a factor of 2x up to 10x performance
improvement on CF8 compared to CF7. I have heard one other person
report really slow CF8 behavior and it was a CF7 -> CF8 upgrade rather
than a clean CF8 install (but other CF7 -> CF8 upgrades have seen the
same performance I've seen).

Can you give more details about your system setup?

--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

[rupesh_kumar]

There is definitely something wrong with the setup. As Sean said, we have consistently seen CF8 performance to be > 2x of CF7.

Did you run both the test on the same machine? What is the OS /VM /RAM? What are the JVM options?

[MikeP]

I agree 100%. The problem is definitely with my setup somewhere. If CF8 really were 6x slower than CF7 there'd be a lot more people besides me complaining I'm sure.

Both tests were run on the same machine. I ran the test on CF8, uninstalled, reinstalled CF7 and ran the test again. The only thing that changed between the tests was the version of CF.

The machine used to run the test (my development machine) is a laptop. It's a Dell Latitude D610 which has a 2GHz Pentium M processor and 2GB of RAM. I'm running on the standard JRun versions that are included with each respective version of CF and I'm using the default JVM as well.

Unfortunately I've been very busy since I made this post and haven't had time to try another machine. I'm working on that now and will report back my results when I do.

Sean, I first did an upgrade from 7 to 8. I saw the same post you were referring to during my troubleshooting efforts so I tried completely uninstalling anything CF related and doing a fresh install of CF8. The slowness persisted.

Thanks for your help so far. I really appreciate it. I'll be back with more info.

[Guest]

MikeP, you're not going mad. We've installed a clean CF8 on 4 separate dev machines (PC, XP SP2, 1Gb RAM) and have seen similar behaviour running the same app under CF7 and CF8. CF7 is significantly faster every time. Under CF7 the app initialises in approx imately 30 secs, and under CF8 the same operation takes over 100 secs.

This application creates large numbers of CFCs via CreateObject and performs much string concatenation via cfsavecontent.

[Guest]

MikeP, would it be possible for you to provide your test harness so we can also try to duplicate and see if there's commonality in our (for example) JVM settings that may be causing this?

[MikeP]

Ok, I finally got a moment to try my test on another machine. I added a bit of code to show some specs about the environment to the top of my test. Here are the results:

CF: ColdFusion Server 8,0,0,176276 Developer
JVM: Java(TM) SE Runtime Environment v1.6.0_01, Sun Microsystems Inc., Spec: 1.6
Heap Memory: allocated 492MB, free 439MB, max 493MB
OS: Windows XP 5.1
---
blank: 0.062 ms
blank: 0.049 ms
blank: 0.047 ms
cfparam: 84.789 ms
component: 40.638 ms
dateFunctions: 189.340 ms
evaluate: 75.864 ms
list: 52.046 ms
REFindAndNoCase: 80.105 ms
REReplaceAndNoCase: 113.880 ms
blank: 0.053 ms
blank: 0.046 ms
blank: 0.046 ms
---
Total Time: 636.965

Wow! Now that's fast! And this machine was a lesser machine than I ran the first two tests on. (Dell Latitude D520, Intel Celeron M 1.73GHz, 1GB Ram). Now I'm going to try re-adding all of my previous configuration options until I find the one that slows it down. I'll report back when I find something.

edgriffiths, I can't send the code as-is because it contains some of my company's proprietary code (the timer stuff.) If I get a minute I'll replace that with the CF built-in timer stuff and send it to you. I'm working on this testing as a side-project so it's tough to find time to devote to it. Please bear with me.

[Guest]

MikeP, thanks for posting this information. I look forward to the results of your future tests.

[Guest]

MikeP, thanks for the update on this. You've given us some hope! We're also looking at our JVM settings so will post if we find anything conclusive.

[Guest]

MikeP, Sean or Rupesh - please could you post the contents of your jvm.config file? Thanks.

[ksmith]

If your CF8 is running slower using IIS, you may want to do some quick tests using the internal JWS webserver. If your speed suddenly increases I suspect you are dealing with a webserver connector issue. Uninstall all connecters by running cfroot\bin\connectors\remove_all_connectors.bat. Confirm all connector traces have been removed from IIS at both the global and individual website level. Then re-run the connector using wsconfig or iis_connector.bat.

[BKBK]

MikeP wrote:
Both scenarios used ...
- Blank application.cfm

Make it Application.cfm. Also, don't leave it blank. What is usually called a blank application file should not necessarily be codeless. Let it at least have minimal code like:

<cfapplication name="testApp"
applicationtimeout="#createtimespan(1,0,0,0)#"
sessionmanagement="yes"
sessiontimeout="#createtimespan(0,0,20,0)#"
setclientcookies="yes">

What happens when you make the comparison now?

[daveb12062933]

Could some of you guys who are not having CF 8 performance problems post your JVM arguments? I'd like to see what is working for others.

[Guest]

looks interesting:

http://corfield.org/blog/index.cfm/do/blog.entry/entry/Java_6_and_ColdFusion_8

http://forum.java.sun.com/thread.jspa?threadID=5218663

[Guest]

Another interesting post, containing JVM config switches:
http://www.alagad.com/go/blog-entry/don-t-throw-coldfusion-out-we-can-help

Has anyone tried the new Java 6 Update 4? Huge number of fixes here:
http://java.sun.com/javase/6/webnotes/ReleaseNotes.html#160_04

[Libby_H]

I too have noticed that my startup of the CF service was taking an extremely long time, and followed the suggestions of the http://www.alagad.com/go/blog-entry/don-t-throw-coldfusion-out-we-can-help website and changed my JVM path from the one provided with CF8 to a previous version.

Question, does anyone know if there are issues reverting to JVM 1.4.2_13? I don't have 1.5 installed, and wanted to use what was available.

[ksmith]

Libby,
Most users would not use startup times as a measure of performance. More important for most is how fast do your applications run. Most users note large improvements in performance using CF8. CF8 and CFMX7 take longer to startup than earlier versions because more services are being started like flex and flash remoting.

Running CF8 on jdk1.4.2 is unsupported. Also features like flex and monitoring will not work.