16 Replies Latest reply on Jan 29, 2008 8:48 AM by ksmith

    CF8 Performance Issues

      Everyone has been raving about how much faster CF8 is than previous versions. This got me very excited at first. Well, I've been running CF8 on my development machine for a couple weeks now. It's been painfully slow compared to CF7, but I've been putting up with it thinking that I must be missing some crucial setting that makes it fast. Well, I've checked every setting, tried every tweak, uninstalled/reinstalled, you name it - but it was still slow.

      So today I decided to run my own crude performance test. I turned off all other software on my machine. I wrote a quick and dirty app, borrowing some code from the app used in the CF8 Performance Whitepaper. I adjusted the number of iterations in the loops so each test area would take about 2000 milliseconds on CF8.

      Instead of measuring the times externally using a program like JMeter, I did a getTickCount() style measurement around the individual tests and outputted the results to the screen. To get the actual times I used a custom timer class I wrote in Java that uses JNI to get a far more accurate (sub-millisecond) timing. If you're tempted to blame my timer, please remember that it was slow performance in general that drove me to this. CF's built-in timing mechanisms reflect the same slowness. I just used my own timer so I could get a more accurate picture.

      Both scenarios used the following server config.
      - Blank application.cfm
      - Trusted cache on
      - Debug off
      - Robust exceptions off
      - All profilers and memory monitors, etc in CF8 were turned off.
      - All non-essential external programs were closed.
      - Verity Search service was disabled.
      - 512MB minimum/maximum JVM heap size.

      I ran ran the test cycle 6 times on both versions of CF. Each test cycle ran three blank tests (inclusion of a blank template) at the start and end as a "control group." Below I took my best CF8 results and compared them with my worst CF7 results.

      Best CF8 Times:
      blank: 0.157 ms
      blank: 0.136 ms
      blank: 0.141 ms
      cfparam: 2,113.305 ms
      component: 2,137.075 ms
      dateFunctions: 1,939.005 ms
      evaluate: 1,791.225 ms
      list: 1,849.384 ms
      REFindAndNoCase: 1,937.942 ms
      REReplaceAndNoCase: 2,609.426 ms
      blank: 0.149 ms
      blank: 0.138 ms
      blank: 0.134 ms

      Total Time: 14,378.217 ms

      Worst CF7 Times:
      blank: 0.081 ms
      blank: 0.071 ms
      blank: 0.070 ms
      cfparam: 301.511 ms
      component: 1,369.429 ms
      dateFunctions: 197.770 ms
      evaluate: 237.860 ms
      list: 68.813 ms
      REFindAndNoCase: 75.692 ms
      REReplaceAndNoCase: 141.153 ms
      blank: 0.043 ms
      blank: 0.030 ms
      blank: 0.030 ms

      Total Time: 2,392.552 ms

      So for me CF7 was about 600% faster than CF8. I'm now back on CF7 and everything is fast again, but I really want to use CF8. I'm looking for suggestions as to why it could have been so slow for me. Have there been any common configuration issues that have caused performance bottlenecks? I have tried all of the obvious stuff. Turning on trusted cache, disabling debugging, giving more memory to the JVM heap, reinstalling, etc. Nothing seems to work. Next I'll probably try it on another machine, but I'm hoping for some suggestions in the mean time.
        • 1. Re: CF8 Performance Issues
          Level 7
          In article <fcbqbe$62k$1@forums.macromedia.com> "MikeP"
          <webforumsuser@macromedia.com> wrote:
          > Everyone has been raving about how much faster CF8 is than previous
          > versions. [big snip]

          Weird. I'm consistently seeing a factor of 2x up to 10x performance
          improvement on CF8 compared to CF7. I have heard one other person
          report really slow CF8 behavior and it was a CF7 -> CF8 upgrade rather
          than a clean CF8 install (but other CF7 -> CF8 upgrades have seen the
          same performance I've seen).

          Can you give more details about your system setup?

          Sean A Corfield -- (904) 302-SEAN
          An Architect's View -- http://corfield.org/

          "If you're not annoying somebody, you're not really alive."
          -- Margaret Atwood

          • 2. Re: CF8 Performance Issues
            rupesh_kumar Adobe Employee
            There is definitely something wrong with the setup. As Sean said, we have consistently seen CF8 performance to be > 2x of CF7.

            Did you run both the test on the same machine? What is the OS /VM /RAM? What are the JVM options?
            • 3. Re: CF8 Performance Issues
              edgriffiths Level 1
              MikeP, you're not going mad. We've installed a clean CF8 on 4 separate dev machines (PC, XP SP2, 1Gb RAM) and have seen similar behaviour running the same app under CF7 and CF8. CF7 is significantly faster every time. Under CF7 the app initialises in approx imately 30 secs, and under CF8 the same operation takes over 100 secs.

              This application creates large numbers of CFCs via CreateObject and performs much string concatenation via cfsavecontent.
              • 4. Re: CF8 Performance Issues
                MikeP Level 1
                I agree 100%. The problem is definitely with my setup somewhere. If CF8 really were 6x slower than CF7 there'd be a lot more people besides me complaining I'm sure.

                Both tests were run on the same machine. I ran the test on CF8, uninstalled, reinstalled CF7 and ran the test again. The only thing that changed between the tests was the version of CF.

                The machine used to run the test (my development machine) is a laptop. It's a Dell Latitude D610 which has a 2GHz Pentium M processor and 2GB of RAM. I'm running on the standard JRun versions that are included with each respective version of CF and I'm using the default JVM as well.

                Unfortunately I've been very busy since I made this post and haven't had time to try another machine. I'm working on that now and will report back my results when I do.

                Sean, I first did an upgrade from 7 to 8. I saw the same post you were referring to during my troubleshooting efforts so I tried completely uninstalling anything CF related and doing a fresh install of CF8. The slowness persisted.

                Thanks for your help so far. I really appreciate it. I'll be back with more info.

                • 5. Re: CF8 Performance Issues
                  edgriffiths Level 1
                  MikeP, would it be possible for you to provide your test harness so we can also try to duplicate and see if there's commonality in our (for example) JVM settings that may be causing this?
                  • 6. Re: CF8 Performance Issues
                    MikeP Level 1
                    Ok, I finally got a moment to try my test on another machine. I added a bit of code to show some specs about the environment to the top of my test. Here are the results:

                    CF: ColdFusion Server 8,0,0,176276 Developer
                    JVM: Java(TM) SE Runtime Environment v1.6.0_01, Sun Microsystems Inc., Spec: 1.6
                    Heap Memory: allocated 492MB, free 439MB, max 493MB
                    OS: Windows XP 5.1
                    blank: 0.062 ms
                    blank: 0.049 ms
                    blank: 0.047 ms
                    cfparam: 84.789 ms
                    component: 40.638 ms
                    dateFunctions: 189.340 ms
                    evaluate: 75.864 ms
                    list: 52.046 ms
                    REFindAndNoCase: 80.105 ms
                    REReplaceAndNoCase: 113.880 ms
                    blank: 0.053 ms
                    blank: 0.046 ms
                    blank: 0.046 ms
                    Total Time: 636.965

                    Wow! Now that's fast! And this machine was a lesser machine than I ran the first two tests on. (Dell Latitude D520, Intel Celeron M 1.73GHz, 1GB Ram). Now I'm going to try re-adding all of my previous configuration options until I find the one that slows it down. I'll report back when I find something.

                    edgriffiths, I can't send the code as-is because it contains some of my company's proprietary code (the timer stuff.) If I get a minute I'll replace that with the CF built-in timer stuff and send it to you. I'm working on this testing as a side-project so it's tough to find time to devote to it. Please bear with me.
                    • 7. Re: CF8 Performance Issues
                      MikeP, thanks for posting this information. I look forward to the results of your future tests.
                      • 8. Re: CF8 Performance Issues
                        edgriffiths Level 1
                        MikeP, thanks for the update on this. You've given us some hope! We're also looking at our JVM settings so will post if we find anything conclusive.
                        • 9. Re: CF8 Performance Issues
                          edgriffiths Level 1
                          MikeP, Sean or Rupesh - please could you post the contents of your jvm.config file? Thanks.
                          • 10. Re: CF8 Performance Issues
                            ksmith Level 1
                            If your CF8 is running slower using IIS, you may want to do some quick tests using the internal JWS webserver. If your speed suddenly increases I suspect you are dealing with a webserver connector issue. Uninstall all connecters by running cfroot\bin\connectors\remove_all_connectors.bat. Confirm all connector traces have been removed from IIS at both the global and individual website level. Then re-run the connector using wsconfig or iis_connector.bat.
                            • 11. CF8 Performance Issues
                              BKBK Adobe Community Professional & MVP
                              MikeP wrote:
                              Both scenarios used ...
                              - Blank application.cfm

                              Make it Application.cfm. Also, don't leave it blank. What is usually called a blank application file should not necessarily be codeless. Let it at least have minimal code like:

                              <cfapplication name="testApp"

                              What happens when you make the comparison now?

                              • 12. Re: CF8 Performance Issues
                                dbeck Level 1
                                Could some of you guys who are not having CF 8 performance problems post your JVM arguments? I'd like to see what is working for others.
                                • 14. Re: CF8 Performance Issues
                                  edgriffiths Level 1
                                  Another interesting post, containing JVM config switches:

                                  Has anyone tried the new Java 6 Update 4? Huge number of fixes here:
                                  • 15. Re: CF8 Performance Issues
                                    Libby H Level 1
                                    I too have noticed that my startup of the CF service was taking an extremely long time, and followed the suggestions of the http://www.alagad.com/go/blog-entry/don-t-throw-coldfusion-out-we-can-help website and changed my JVM path from the one provided with CF8 to a previous version.

                                    Question, does anyone know if there are issues reverting to JVM 1.4.2_13? I don't have 1.5 installed, and wanted to use what was available.
                                    • 16. Re: CF8 Performance Issues
                                      ksmith Level 1
                                      Most users would not use startup times as a measure of performance. More important for most is how fast do your applications run. Most users note large improvements in performance using CF8. CF8 and CFMX7 take longer to startup than earlier versions because more services are being started like flex and flash remoting.

                                      Running CF8 on jdk1.4.2 is unsupported. Also features like flex and monitoring will not work.