1 2 3 Previous Next 85 Replies Latest reply: Dec 22, 2014 12:17 PM by Anit_Kumar RSS

    ColdFusion 10 intermittent "service not available"

    Ripples Community Member

      Have been programming over 30 years and CF in its various forms since 1998, but have to say this upgrade is NOT straightforward and after 3 very frustrating days I've finally reverted to CF9.

       

      CF10 Standard upgrade from CF9. Running on dedicated IIS 7.5 on WIN 2008 R2 Server. All 64bit.

       

      Upgrade installation worked correctly. RDS was selected as was the Upgrade all IIS sites.

       

      Initially CF10 failed to start and checking IIS showed the connectors had NOT been installed. CF10 Administrator just showed raw machine code.

       

      Used the CF10 Web Server Configuration tool on "ALL Sites" - no difference.

       

      Removed "All sites" and then installed the connectors against individual sites. This worked - but then I noticed each site was running the CF9 connectors!

       

      Removed each. Then ran the CF9 Web Server Configuration tool to remove all CF9 connectors. Confirmed they had been removed in IIS, then ran CF10 Web Server Configuration tool to add the connectors and all worked well.

       

      This morning I rebooted the server to confirm CF had "installed" correctly, 3 of my 5 sites too 10 minutes before they could be accessed. Then after 30 minutes I'd get a simple white screen with "service unavailable".

       

      Can't afford to have our sites and clients sites intermittent so have reverted to the CF9 connectors and took CF10 offline.

       

      Hopefully someone else has had similar experience and can shed some light on this issue.

       

      Thanks

      Peter

        • 1. Re: ColdFusion 10 intermittent "service not available"
          Ripples Community Member

          Update for others hitting issues with CF10 and IIS on Win 2008 R2 64 Bit.....

           

          Thought I'd try a clean install and not an upgrade to CF9. I restored the server boot drive to a pre CF10 4installation state. Unistalled all connectors then unistalled CF9. Rebooted, then installed a clean version of CF10.  Hit some IIS issues as the default document and cfm connectors did not flow down to "all sites". I corrected by 'resetting to parent' for these settings. Then manually configured settings in CF Admin.

           

          All worked correctly UNTIL I rebooted and then hit the "service not available" again with a footnote of  "jakarta/isapi/isapi_redirector/1.2.32"

           

          Searching Google I found other users hitting the same issue in that this Tomcat connector fails to load normally on IIS start. Not sure if its a 64bit issue, but as I mentioned this server is running 64bit Win 2008R2, IIS 7.5 and CF10 64 bit. I'd have thought this was a very common server configuration.

           

          CF9 64 bit had no issues.

           

          Sadly I see no responses from Adobe.

           

          Have given up with CF10.

           

          Hopefully Adobe can come up with a fix soon!

          • 2. Re: ColdFusion 10 intermittent "service not available"
            Stephen.Walker Community Member

            Had similar issues and attempted install/reinstall.  Found that the Handler Mappings were completely wrong.  Changed them to point to the correct location and all the problems disappeared. 

            • 3. Re: ColdFusion 10 intermittent "service not available"
              Ripples Community Member

              Thanks Stephen!

               

              I also found mappings were still pointing at CF9 connectors after the standrad installtion and changed them using the CF10 Connector Tool. Confirmed they were correctly pointing to CF10 connectors and all worked well until a reboot when the connectors intermittently "dropped out" and I get the "service unavailable" screen.

              • 4. Re: ColdFusion 10 intermittent "service not available"
                Ripples Community Member

                Update 6/20/2012

                 

                Have been troubleshooting this connector issue daily unfortunately still without success.

                 

                Have tried the upgrade from CF9 to CF10. Removed CF9 completely and installed a fresh copy of CF10. All with the same intermittent issue that the connector drops out on some web sites after a reboot. I have confirmed in IIS that the default documents include index.cfm and the handlers for CFM etc are pointing to the new Tomcat redirect connector.

                 

                Sites work perfectly once setup, but one or two will drop out after every reboot. Mostly they just hang with a blank page showing. Sometimes I'll get "service unavailable".

                 

                If I look at my CF logs there no error messages. If I look at my IIS logs it just shows a huge delivery time while I get the connector renewed for that site.

                 

                There's only 4 sites on this server and the failed ones can be any. No consistency there.

                 

                Have tried removing all connectors in CF9 before installing CF10 and running the CF10 connector tool but still get the same results.

                 

                Tried installing the All Sites connector - same result. Tried individual site connectors with and without the All Sites Connector - same result.

                 

                Have confirmed I can get the failed site up and running by using the connector tool to remove the bad site connector and then re-install it. Site remain good until a reboot.

                 

                Restarting the WWW Publishing Service does not cause any problems

                 

                Restarting all CCF10 Services also does not cause any problems.

                 

                Install was from ColdFusion_10_WWEJ_win64.exe dated 5/3/2012 that came direct from Adobe on disc - so can't blame corrupt download.

                 

                As I mentioned before, I know CF very well having been programming full time with it since 1998. I always keep the server updated to the latest software and while there's always quirks to work around, I confess this one is proving very frustrating.

                 

                Hopefully Adobe will step in soon!

                 

                Peter

                • 5. Re: ColdFusion 10 intermittent "service not available"
                  Stephen.Walker Community Member

                  Knock on wood, but I haven't experienced this issue yet.  My guess is that it is more the Apache integration.  Are all the services restarting?  If this is happening repeatedly, I would file a bug over at https://bugbase.adobe.com/

                  • 6. Re: ColdFusion 10 intermittent "service not available"
                    Ripples Community Member

                    Thanks again Stephen!

                     

                    I did raise a bug report #3216317

                     

                    First response said it could not be reproduced.

                     

                    I did find a similar issue at http://forums.adobe.com/message/4465029 I also see lots of "nul null error in the coldfusion-error.log but seems to be a different cause

                    +++++++++++++++++++++

                    Jun 20, 2012 1:19:54 PM org.apache.tomcat.util.http.Parameters processParameters

                    INFO: Invalid chunk starting at byte [9] and ending at byte [9] with a value of [null] ignored

                    +++++++++++++++++++++++++++++++++++

                     

                    However these errors are continuous whereas the site stall occurs immediately after a reboot and is cleared by rerunning the connector tool for CF10 on the bad site.

                    • 7. Re: ColdFusion 10 intermittent "service not available"
                      Ripples Community Member

                       

                      UPDATE 8/3/2012

                      I spent the past week trying to troubleshoot this issue in a little more depth.

                      Have used the Web Server Configuration Tool in all combinations of "ALL" or individual sites or both. If I remove all connectors and start afresh, selecting "ALL" does not always flow down the CF file Handlers to the individual sites. Only way this seemed to work is if I reset the handlers for each individual site to "revert to parent".

                      However, even when connectors could be confirmed to exist correctly through IIS Manager, this intermittent failure for all sites to start after a reboot continued.

                      After much testing I've found I can resolve the problem every time by waiting a couple of minutes after a reboot and then restarting the IIS Service. Subsequently the server works well. CF10 is fast and I see no issues.

                      I did find a couple of entries in the C:\CF10\cfusion\logs\coldfusion-error.log that may offer a clue

                      +++++++++++++++++++++++++++++++++++++++++++++++++++

                       

                      During CF Start

                      several....

                      *******************************

                       

                      Jul 31, 2012 1:17:51 PM org.apache.tomcat.util.http.Parameters processParameters

                      INFO: Invalid chunk starting at byte [0] and ending at byte [0] with a value of [null] ignored

                      ********************************

                       

                      and....

                      *********************************

                       

                      Jul 31, 2012 1:22:23 PM org.apache.catalina.core.AprLifecycleListener init

                      INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\\CF10\\cfusion\lib;C:\\CF10\\cfusion\jintegra\bin;C:\\CF10\\cfusion\jintegra\bin\inter national;C:\\CF10\\cfusion\lib\oosdk\classes\win

                      **********************************

                       

                      and....

                      **********************************

                       

                      Jul 31, 2012 1:22:36 PM com.adobe.coldfusion.launcher.Launcher run

                      INFO: Server startup in 14761 ms

                      Error: Could not load mediaLib accelerator wrapper classes. Continuing in pure Java mode.

                      Occurs in: com.sun.media.jai.mlib.MediaLibAccessor

                      com.sun.media.jai.mlib.MediaLibLoadException

                      ***********************************

                       

                      and several times.....

                      ***********************************

                       

                      Jul 31, 2012 2:15:47 PM org.apache.coyote.ajp.AjpProcessor getRealPathFromServer

                      SEVERE: Error in getRealPathFromConn

                      java.net.SocketException: Connection reset by peer: socket write error

                      at java.net.SocketOutputStream.socketWrite0(Native Method)

                      at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)

                      at java.net.SocketOutputStream.write(SocketOutputStream.java:124)

                      at org.apache.coyote.ajp.AjpProcessor.getRealPathFromServer(AjpProcessor.java:391)

                      at org.apache.coyote.ajp.AbstractAjpProcessor.getRealPath(AbstractAjpProcessor.java:1112)

                      at org.apache.naming.resources.FileDirContext.doGetRealPath(FileDirContext.java:198)

                      at org.apache.naming.resources.BaseDirContext.getRealPath(BaseDirContext.java:412)

                      at org.apache.catalina.core.StandardContext.getRealPath(StandardContext.java:4475)

                      at org.apache.catalina.core.ApplicationContext.getRealPath(ApplicationContext.java:414)

                      at org.apache.catalina.core.ApplicationContextFacade.getRealPath(ApplicationContextFacade.ja va:335)

                      at coldfusion.runtime.ServletContextWrapper.doGetRealPath(ServletContextWrapper.java:179)

                      at coldfusion.runtime.ServletContextWrapper._doGetRealPath(ServletContextWrapper.java:116)

                      at coldfusion.runtime.ServletContextWrapper.getRealPath(ServletContextWrapper.java:98)

                      at coldfusion.runtime.TemplateProxyFactory.getFullName(TemplateProxyFactory.java:1092)

                      at coldfusion.runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:202)

                      at coldfusion.runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:159)

                      at coldfusion.runtime.TemplateProxyFactory.resolveFile(TemplateProxyFactory.java:120)

                      at coldfusion.cfc.CFCProxy.<init>(CFCProxy.java:138)

                      at coldfusion.cfc.CFCProxy.<init>(CFCProxy.java:84)

                      at coldfusion.runtime.AppEventInvoker.<init>(AppEventInvoker.java:64)

                      at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:226)

                      at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)

                      at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)

                      at coldfusion.filter.PathFilter.invoke(PathFilter.java:112)

                      at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:94)

                      at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:2 8)

                      at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)

                      at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)

                      at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)

                      at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)

                      at coldfusion.filter.CachingFilter.invoke(CachingFilter.java:62)

                      at coldfusion.CfmServlet.service(CfmServlet.java:204)

                      at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)

                      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.j ava:305)

                      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)

                      at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42 )

                      at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)

                      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.j ava:243)

                      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)

                      at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)

                      at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)

                      at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)

                      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)

                      at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)

                      at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:928)

                      at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)

                      at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:414)

                      at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:203)

                      at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.jav a:539)

                      at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:298)

                      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

                      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

                      at java.lang.Thread.run(Thread.java:662)

                      ***************************************

                      I

                      • 8. Re: ColdFusion 10 intermittent "service not available"
                        carl type3 Community Member

                        Re Durring CF start:
                        org.apache.catalina.core.AprLifecycleListener init
                        INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\\CF10\\etc

                         

                        Was discussed in this post:

                        http://forums.adobe.com/message/4446569#4446569

                         

                        Maybe the details in that post to resolve that message will partly provide solution to your issue.

                         

                        HTH, Carl.

                        • 9. Re: ColdFusion 10 intermittent "service not available"
                          Ripples Community Member

                          Thanks Carl,

                           

                          I appreciate the pointer. I did come across the post during my search for any clues, but following the conclusion that the library wasn't necessary I decided to not to complicate the CF10 install.

                           

                          My current concern was the "SEVERE: Error in getRealPathFromConn" entries which I'm hoping someone out there may know what these are.

                           

                          Thanks again

                          Peter

                          • 10. Re: ColdFusion 10 intermittent "service not available"
                            carl type3 Community Member

                            Peter I recall seeing Jrun (CF6 7 8?) and getRealPathFromConn errors before but have not directly been involved with matter so it does not come to mind as to solution. Perhaps if you search without tomcat details you might find a useful lead since CF has run Jrun for longer than tomcat.

                             

                            HTH again, Carl.

                            • 11. Re: ColdFusion 10 intermittent "service not available"
                              Ripples Community Member

                              Thanks Carl a good lead... Coding error on undeclared variable caused the "SEVERE: Error in getRealPathFromConn" entry in the log. Despite clearing this error the failure of all sites to start after reboot persists. CF Logs show no errors duing or after the latest reboot. On the last reboot 2 out of 4 sites 'hung'. Tried the usual IIS service restart, but 1 site still hung. Shut down IIS service then restarted CF10 service which brought all 4 sites back up.

                               

                              I do note that stopping the CF10 service takes considerably longer than previous CF version. Is this normal?

                               

                              Peter

                              • 12. Re: ColdFusion 10 intermittent "service not available"
                                Ryan Luo Community Member

                                Hi Peter

                                Have you figured the way to resolve your issue with cf10 yet? Or you have reverted back to cf 9. It seems we may face the similar issue. Can you please check your system error log to see whether you got some similar erroes as I did. I have posted the detail in

                                http://forums.adobe.com/message/4609125

                                My production server is really unstable at the moment and this forcing me to monitor the server 24/7 in case it hangs.

                                • 13. Re: ColdFusion 10 intermittent "service not available"
                                  Ripples Community Member

                                  Ryan,

                                   

                                  Sorry for the slow response...

                                   

                                  After several weeks trying to resolve this CF10 problem full time the final staw came when the Tomcat Service crashed twice in one day leaving my customers and clients staring at an error page. Not to mention the loss in revenue.

                                   

                                  I therefore restored CF9 earlier this month and am once again running on a stable configuration.

                                   

                                  I received no response from Adobe as to what may be the cause, but would warn anybody considering upgrading to CF10 on a 64 bit Windows OS. It could be that a clean install is fine, but at this stage I can't see the improvement over CF9 worth the disruption of starting with a clean OS.

                                   

                                  As I previously mentioned, I've been running CF since 1998 and this is the first issue I've been unable to resolve.

                                   

                                  Peter

                                  • 14. Re: ColdFusion 10 intermittent "service not available"
                                    Ryan Luo Community Member

                                    Peter,

                                    Thank you for your response.

                                    There are two bugs which might associate to our issue have been submitted to Adobe and I believe they are all confirmed by Coldfusion Team. As far as I know, they are working on it at the moment, but unfortunately they are unable to put the fix in hotfix 1 they have released early this month or late last month.

                                    https://bugbase.adobe.com/index.cfm?event=bug&id=3318104

                                    https://bugbase.adobe.com/index.cfm?event=bug&id=3222748

                                    We are waiting for the fix from Adobe, since we are not going to rollback to where we came from.

                                    My temperate solution is increasing the number of error IIS can handle before shut itself off.

                                    It is in application pool > Advance setting > Rapid-Faill Protection - Maximum Failures

                                    Since I have found the IIS will shut itself off after 5 errors been sent from connector, by increasing the number here, it temperately solved my problem (but my event log still full of errors from connector)

                                     

                                    Ryan

                                    • 15. Re: ColdFusion 10 intermittent "service not available"
                                      charlie@carehart.org Community Member

                                      Peter, I hear your frustration, and I have some thoughts.

                                       

                                      I don't see anyone suggesting this yet: is it possible that this is all down to a permissions issue? I realize that at this point, you have likely uninstalled CF10 and may not care to futz with it anymore, but in case you may, or in case others may find this, here are a couple of suggestions.

                                       

                                      First, did you notice how things would be ok until you rebooted? That leads me to think that something that was "changed" was really only changed in memory and could not be written to disk, so that on reboot, the "change" was "lost". I can't right now think what that may have been, but it sounds like a clue worth considering.

                                       

                                      One thing I do know is that it may help when running the web server configuration tool if you first right-click on it and "run as administrator". This is one of those vagaries that arises from the increased security in Windows 2008. I do think that sometimes CF is getting an unfair black eye due to things like this.There are other apps which work funky if you don't "run as admin". Some even give no error, but still don't do what you think they should do.

                                       

                                      Also, there are important differences in IIS 7 from how things worked before, which could also be related (assuming you are not using IIS 6 Compatibily mode). Some folks may have noticed that a web.config file may be written to the root of your IIS sites which have been configured for CF. That could be created by changes you make in the IIS interface, or by the CF web server config tool (or installer).

                                       

                                      Well, as for the latter here's another potential gotcha: perhaps CF doesn't have permissions to that directory. For those who change the user under which CF runs, they may not consider any need to give write permissions for that user to their website docroot. If CF can't write to it, then I could see the web server config tool (or installer) not implementing things correctly.

                                       

                                      Not all configurations of CF and IIS require creation of the web.config file, so I do realize some will say they see none there and yet things are working. Put briefly, the differences have to do with if you are or are not running IIS in IIS 6 compatibility mode. If you are not, then it works quite differently than it did before.

                                       

                                      First, it seems that if you configure "all sites", CF configures things (like what you see in handler mappings) in an applicationhost.config file in \Windows\System32\inetsrv\config, and any changes from that made for a given site are written to its web.config. And second, if you configure a specific site, then it writes the needed settings to that site's web.config. In either case, again, if there are any challenges writing to that file, then settings won't be persisted. I grant that I'm still guessing a bit here, so I'll welcome thoughts from others.

                                       

                                      BTW, this use of the web.config could also explain some of the confusion you were hitting when you had things configured for both CF 9 and CF10. Perhaps you may have had (as I did at one time) a site set to connect using CF9, and another set to connect using CF10, both pointing to the same web site docroot. That was always ok in CF traditionally, to test things on one server with different sites against different versions of CF.

                                       

                                      But now (if not using IIS 6 Compat mode) then CF and IIS are using these config files, and they are NOT writing different ones for different versions. If 2 sites point to the same docroot, changes to each basically overwrite the other. It can definitely be confusing. 

                                       

                                      And for many people, CF10 is the first release where they are deploying CF on IIS 7. You may know that CF 9.0 and below did not formally support it, but people found they could get it to work by enabling IIS 6 compatibility. And while 9.0.1 added full support for IIS 7, many just continued to use IIS 6 Compat when they upgraded from 9.0 to 9.0.1. I suspect now we have more people going to IIS 7 for the first time with 10, or maybe putting it on new boxes where they did not think to even turn on IIS 6 compat, and so they are experiencing the results of this new approach to things for the first time.

                                       

                                      Again, some of what I offer here is speculation. A lot of this is new for all of us. I just wanted to offer these thoughts as I'd not see anyone else do so yet here. I appreciate the severe annoyance some have felt in things seeming "not to work". But I'll just say that there are people who are running CF10 fine, so it is possible to configure things correctly.

                                       

                                      This is somewhat like the pain we went through in the move from CF 5 to CFMX (6), where things with Java were quite different and some even gave up on CF (or just never moved up past 5, or not for many years.) Then there's the added differene that CF10 is running on Tomcat rather than JRun, and Adobe did have to create a new connector (and couldn't use the stock one that comes with Tomcat). With so many moving parts, it can be hard to know for sure when something is a "bug", when something is just a misconfiguration, or a misunderstanding, or a failure to set proper permissions, or to run as administrator, and on and on.

                                       

                                      As in that time fo the switch to CFMX, there may be growing pains but there will also be growing understanding as more and more people find and resolve problems such as these, and share their knowledge. It's little consolation for those who are on the front lines deploying the new release in production.

                                       

                                      But I hope, Peter, that perhaps some of what I've shared here might either help solve problems or at least help get you further down the road to resolving them, again, if and when you may be willing to give deployment of CF10 another shot.

                                      • 16. Re: ColdFusion 10 intermittent "service not available"
                                        Ripples Community Member

                                        Hi Charlie,

                                         

                                        Thanks for the detailed response. Well written and very much appreciated.

                                         

                                        I do agree that the change between CF9 and 10 is similar to the introduction of CFMX. The complexities of getting this to work across different operating systems must be daunting task to the developer.

                                         

                                        I've been messing with code long enough to remember the introduction of PC DOS, MS DOS and Windows. Took a few years before processing power made Windows usable, but I've stuck with that OS in its various forms from those early days. I try to take advantage of the latest OS and software versions wherever possible.

                                         

                                        Similarly I've been with CF since '98 (version 2?). Upgrades have gone smoothly, but occaisionally a new server has given me the chance to install a fresh copy onto a fresh copy of Windows and SQL. Normally this would be my fix for this issue - upgrade to new server hardware and carry out a fresh install. Unfortunately time and costs mean this is not currently an option hence sticking with trying to fix this frustrating error for so long.

                                         

                                        At the time of upgrade I was running the latest CF9 (9.0.1.274733). Yes, I was in IIS6 compatibility mode, and feel permissions were fine.

                                         

                                        Just to clarify, after any reboot 1 or more of my 4 commercial sites would hang until the Tomcat error page appeared. Knowing this I'd check each site immediately after a reboot, then reboot IIS and occaisionally CF10 until all  4 sites appeared.

                                         

                                        Initially I found the WEB Configuration Tool made a complete mess of the web.config files, so ended up using the tool to remove all sites and configuring individual but they were still very messy. Eventually I closed down CF10, and spent an hour or so manually configuing all sites in IIS to "revert to parent". Then I used the tool to configure the ALL SITES option. This produced much cleaner web.config files, but didn't resolve the problem.

                                         

                                        Once I got all 4 sites running then they remained stable until the next reboot. Except that is for that last day when the Tomcat Server stopped twice withing 24 hours.

                                         

                                        Yes, I did have CF9 and CF10 running in parallel initially, but during the troubleshooting, I completely uninstally CF10 AND CF9, before installing a fresh copy of CF10. Unfortunately this didn't help.

                                         

                                        I have reverted to CF9 and will stick with it until I see sound evidence that CF10 is stable.

                                         

                                        I do feel it's a connector issue between CF10/Tomcat and IIS. I'm on the 64 bit versions so can't speak for a 32 bit setup.

                                         

                                        Again, many thanks for your response.

                                         

                                        Best wishes

                                        Peter

                                        • 17. Re: ColdFusion 10 intermittent "service not available"
                                          cf_chris Community Member

                                          Ryan,

                                           

                                          Just wanted to confirm that we experienced the issues with IIS listed in those bugs. I was able to get Adobe's attention via social media and we are testing out new jars that fix the issue with the isapi connector.

                                           

                                          We ran for almost 24 hours before having to roll back to our CF9 servers on Friday. (We had to rollback because of this DateConvert() bug.) The new jars appear to fix the connector issue. Hopefully there will be an update soon to address this. But, since the instructions from adobe included dissconnecting sites and running wsconfig after replacing the wsconfig.jar, I'm not sure that a regular update will fix it automatically.

                                           

                                          If you want to test the new jars, Rupesh has an offer to send the jars to people who would test them on this bug page in the notes. (rukumar (at) adobe (dot) com)

                                           

                                          We have again added the CF10 servers into the pool and they are ramping up. Hope to be fully on CF10 by Tuesday.

                                          • 18. Re: ColdFusion 10 intermittent "service not available"
                                            Ryan Luo Community Member

                                            Hi Chris

                                            Thanks for your reply and the information you have provided.

                                            I am afraid I couldn't try the new jar due to the server is in production. I will wait the official patch for cf10 and hopefully it can resolve the issue troubles the community.

                                             

                                            Thanks

                                            Ryan

                                            • 19. Re: ColdFusion 10 intermittent "service not available"
                                              colophonnewmedia Community Member

                                              Ripples, sorry to hear your pain.

                                               

                                              We upgrade to CF 10 running Windows 2008 servers with IIS 7, connecting to MySQL 5 DBs. We put our highest profile customers on these new boxes and promised them a great, upgraded service. We tested, and tested each site was then put live on the new VPSs. Site failures became rampant. IIS got hung up several times a day. Restarting IIS solves the problem, but it is an absurd problem for a middle ware that is on version 10.

                                               

                                              Seems that if there is only 1 site running there is no problem, but if mulitple sites run on the same server, IIS gets hung up.

                                               

                                              I am glad to read that going back to CF 9 solved your problems. This is what we may do.

                                               

                                              Charlie Arehart, above has some good suggestions it seems, and from what I can tell, it is really a matter of CF10 working propertly with IIS 7. Problem can be solved by running IIS 7 with IIS 6 compatibility enabled?

                                               

                                              If anyone has come up with a stable config of the above specs - would love to hear it

                                               

                                              Thanks,

                                              james

                                              • 20. Re: ColdFusion 10 intermittent "service not available"
                                                cf_chris Community Member

                                                James,

                                                 

                                                Update 4 had the connector fix in it. If you apply Update 4 and use wsconfig to disconnect and reconnect your instances to IIS 7, you should be good to go. The crashing should stop.

                                                • 21. Re: ColdFusion 10 intermittent "service not available"
                                                  colophonnewmedia Community Member

                                                  Chris,

                                                   

                                                  Thank you for the note. We are have applied this fix, and so far so good.

                                                   

                                                  We also had to update a setting in CF admin setting for: “Maximum number of simultaneous Template requests”. The default for CF 10 is set to 25. The default install of our local copy of CF 10 has this set to 25. Our box was set to 10, as it was upgraded from 8 and 9 and we think this was left over from CF 8 and 9.

                                                   

                                                  However, according to Adobe, when the limit for Template requests is reached, the system is supposed to queue the rest. In our case it does not queue, it stops. So there is still some concern. 

                                                   

                                                  We also don’t see how a request is equal to site activity for 1 site. We thought that “worker processes” would be related to a number of different processes that happened when web templates run.

                                                  • 22. Re: ColdFusion 10 intermittent "service not available"
                                                    Lee Bartelme Community Member

                                                    Just updated Coldfusion to latest version (Hotfix 7). We are still having this issue. System runs for a week or so and then our main application pool dies with this error over and over:

                                                     

                                                    SEVERE: Error in getRealPathFromConn
                                                    java
                                                    .net.SocketException: Connection reset
                                                        at java
                                                    .net.SocketInputStream.read(SocketInputStream.java:168)
                                                    at org
                                                    .apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:319)
                                                    at org
                                                    .apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:445)
                                                    at org
                                                    .apache.coyote.ajp.AjpProcessor.readString(AjpProcessor.java:425)
                                                    at org
                                                    .apache.coyote.ajp.AjpProcessor.getRealPathFromServer(AjpProcessor.java:407)
                                                    at org
                                                    .apache.coyote.ajp.AbstractAjpProcessor.getRealPath(AbstractAjpProcessor.java:1154)
                                                    at org
                                                    .apache.naming.resources.FileDirContext.doGetRealPath(FileDirContext.java:198)
                                                    at org
                                                    .apache.naming.resources.BaseDirContext.getRealPath(BaseDirContext.java:412)
                                                    at org
                                                    .apache.catalina.core.StandardContext.getRealPath(StandardContext.java:4475)
                                                    at org
                                                    .apache.catalina.core.ApplicationContext.getRealPath(ApplicationContext.java:414)
                                                    at org
                                                    .apache.catalina.core.ApplicationContextFacade.getRealPath(ApplicationContextFacade.java:335)
                                                    at coldfusion
                                                    .runtime.ServletContextWrapper.doGetRealPath(ServletContextWrapper.java:179)
                                                    at coldfusion
                                                    .runtime.ServletContextWrapper._doGetRealPath(ServletContextWrapper.java:116)
                                                    at coldfusion
                                                    .runtime.ServletContextWrapper.getRealPath(ServletContextWrapper.java:98)
                                                    at coldfusion
                                                    .runtime.TemplateProxyFactory.getFullName(TemplateProxyFactory.java:1092)
                                                    at coldfusion
                                                    .runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:202)
                                                    at coldfusion
                                                    .runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:159)
                                                    at coldfusion
                                                    .runtime.TemplateProxyFactory.resolveFile(TemplateProxyFactory.java:120)
                                                    at coldfusion
                                                    .cfc.CFCProxy.<init>(CFCProxy.java:138)
                                                    at coldfusion
                                                    .cfc.CFCProxy.<init>(CFCProxy.java:84)
                                                    at coldfusion
                                                    .runtime.AppEventInvoker.<init>(AppEventInvoker.java:64)
                                                    at coldfusion
                                                    .filter.ApplicationFilter.invoke(ApplicationFilter.java:226)
                                                    at coldfusion
                                                    .filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
                                                    at coldfusion
                                                    .filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
                                                    at coldfusion
                                                    .filter.PathFilter.invoke(PathFilter.java:112)
                                                    at coldfusion
                                                    .filter.ExceptionFilter.invoke(ExceptionFilter.java:94)
                                                    at coldfusion
                                                    .filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)
                                                    at coldfusion
                                                    .filter.BrowserFilter.invoke(BrowserFilter.java:38)
                                                    at coldfusion
                                                    .filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
                                                    at coldfusion
                                                    .filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
                                                    at coldfusion
                                                    .filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
                                                    at coldfusion
                                                    .filter.CachingFilter.invoke(CachingFilter.java:62)
                                                    at coldfusion
                                                    .CfmServlet.service(CfmServlet.java:219)
                                                    at coldfusion
                                                    .bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
                                                    at org
                                                    .apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
                                                    at org
                                                    .apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
                                                    at coldfusion
                                                    .monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42)
                                                    at coldfusion
                                                    .bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
                                                    at org
                                                    .apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
                                                    at org
                                                    .apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
                                                    at org
                                                    .apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
                                                    at org
                                                    .apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
                                                    at org
                                                    .apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
                                                    at org
                                                    .apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
                                                    at org
                                                    .apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
                                                    at org
                                                    .apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:928)
                                                    at org
                                                    .apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
                                                    at org
                                                    .apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:414)
                                                    at org
                                                    .apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:204)
                                                    at org
                                                    .apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)
                                                    at org
                                                    .apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:298)
                                                    at java
                                                    .util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
                                                    at java
                                                    .util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
                                                    at java
                                                    .lang.Thread.run(Thread.java:662)

                                                     

                                                    While the server error is the same each time, the trace is different with each one. The first one always looks like the above, and the rest of them after it look similar to the below where the connection is reset by peer. However the full traces and paths to different files changes with each error usually. Sometimes three or four in a row of the same, sometimes just one. Either way, there are like 40 or 50 in a row.

                                                     

                                                    Jan 25, 2013 8:10:04 AM org.apache.coyote.ajp.AjpProcessor getRealPathFromServer
                                                    SEVERE: Error in getRealPathFromConn
                                                    java.net.SocketException: Connection reset by peer: socket write error
                                                              at java.net.SocketOutputStream.socketWrite0(Native Method)
                                                              at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
                                                              at java.net.SocketOutputStream.write(SocketOutputStream.java:124)
                                                              at org.apache.coyote.ajp.AjpProcessor.getRealPathFromServer(AjpProcessor.java:401)
                                                              at org.apache.coyote.ajp.AbstractAjpProcessor.getRealPath(AbstractAjpProcessor.java:1154)
                                                              at org.apache.naming.resources.FileDirContext.doGetRealPath(FileDirContext.java:198)
                                                              at org.apache.naming.resources.BaseDirContext.getRealPath(BaseDirContext.java:412)
                                                              at org.apache.catalina.core.StandardContext.getRealPath(StandardContext.java:4475)
                                                              at org.apache.catalina.core.ApplicationContext.getRealPath(ApplicationContext.java:414)
                                                              at org.apache.catalina.core.ApplicationContextFacade.getRealPath(ApplicationContextFacade.java:335)
                                                              at coldfusion.runtime.ServletContextWrapper.doGetRealPath(ServletContextWrapper.java:184)
                                                              at coldfusion.runtime.ServletContextWrapper._doGetRealPath(ServletContextWrapper.java:116)
                                                              at coldfusion.runtime.ServletContextWrapper.getRealPath(ServletContextWrapper.java:98)
                                                              at coldfusion.runtime.RuntimeServiceImpl.getRealPath(RuntimeServiceImpl.java:1046)
                                                              at coldfusion.runtime.RuntimeServiceImpl.getRealPath(RuntimeServiceImpl.java:1060)
                                                              at coldfusion.filter.FusionContext.getRealPath(FusionContext.java:858)
                                                              at coldfusion.filter.FusionContext.getRealPath(FusionContext.java:830)
                                                              at coldfusion.runtime.RuntimeServiceImpl.resolveTemplatePath(RuntimeServiceImpl.java:794)
                                                              at coldfusion.runtime.TemplateProxyFactory.getResolvedFile(TemplateProxyFactory.java:1263)
                                                              at coldfusion.runtime.TemplateProxyFactory.getTemplateFileHelper(TemplateProxyFactory.java:1571)
                                                              at coldfusion.cfc.ComponentProxyFactory.getProxy(ComponentProxyFactory.java:51)
                                                              at coldfusion.runtime.CFPage.CreateObject(CFPage.java:5560)
                                                              at coldfusion.runtime.CFPage.CreateObject(CFPage.java:5575)
                                                              at cfApplication2ecfc851739670.runPage(D:\www\facilitator\Application.cfc:30)
                                                              at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:244)
                                                              at coldfusion.filter.SilentFilter.invoke(SilentFilter.java:47)
                                                              at coldfusion.runtime.TemplateProxyFactory.resolveComponentHelper(TemplateProxyFactory.java:538)
                                                              at coldfusion.runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:234)
                                                              at coldfusion.runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:159)
                                                              at coldfusion.runtime.TemplateProxyFactory.resolveFile(TemplateProxyFactory.java:120)
                                                              at coldfusion.cfc.CFCProxy.<init>(CFCProxy.java:138)
                                                              at coldfusion.cfc.CFCProxy.<init>(CFCProxy.java:84)
                                                              at coldfusion.runtime.AppEventInvoker.<init>(AppEventInvoker.java:64)
                                                              at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:226)
                                                              at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
                                                              at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
                                                              at coldfusion.filter.PathFilter.invoke(PathFilter.java:112)
                                                              at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:94)
                                                              at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)
                                                              at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
                                                              at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
                                                              at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
                                                              at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
                                                              at coldfusion.filter.CachingFilter.invoke(CachingFilter.java:62)
                                                              at coldfusion.CfmServlet.service(CfmServlet.java:219)
                                                              at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
                                                              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
                                                              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
                                                              at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42)
                                                              at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
                                                              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
                                                              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
                                                              at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
                                                              at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
                                                              at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
                                                              at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
                                                              at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
                                                              at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:928)
                                                              at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
                                                              at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:414)
                                                              at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:204)
                                                              at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)
                                                              at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:298)
                                                              at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
                                                              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
                                                              at java.lang.Thread.run(Thread.java:662)
                                                    

                                                    This is a critical error and is really bugging the crap out of me. Strange thing is, as far as I know, this has not happened until the last month or so. We've been running CF10 on IIS7 for about 6 months. The first appearance of the error is on Aug 7, however it only fired once. The on Nov 5 once. Then Jan 10 twice. Then on Jan12 it started firing the error repeatedly in a row causing IIS to stop the pool. I updated CF shortly after that to attempt to fix the issue, but it remains an issue. Also, this error has occured 0 times in our development machine. The machines are for all intents and purposes identical. (HDD/RAM is different but nothing else)

                                                     

                                                    The only other thing I can think of, you can slap me for my ignorance, is there are other things in the log file:

                                                     

                                                    A bunch of these:

                                                     

                                                    Jan 18, 2013 8:13:59 AM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
                                                    SEVERE: The web application [/] created a ThreadLocal with key of type [coldfusion.util.DateUtils$1] (value [coldfusion.util.DateUtils$1@cde21de]) and a value of type [java.util.GregorianCalendar] (value [java.util.GregorianCalendar[time=1613372400000,areFieldsSet=true,areAllFieldsSet=true,lenient=false,zone=sun.util.calendar.ZoneInfo[id="America/Denver",offset=-25200000,dstSavings=3600000,useDaylight=true,transitions=157,lastRule=java.util.SimpleTimeZone[id=America/Denver,offset=-25200000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]],firstDayOfWeek=1,minimalDaysInFirstWeek=1,ERA=1,YEAR=2021,MONTH=1,WEEK_OF_YEAR=8,WEEK_OF_MONTH=3,DAY_OF_MONTH=15,DAY_OF_YEAR=46,DAY_OF_WEEK=2,DAY_OF_WEEK_IN_MONTH=3,AM_PM=0,HOUR=0,HOUR_OF_DAY=0,MINUTE=0,SECOND=0,MILLISECOND=0,ZONE_OFFSET=-25200000,DST_OFFSET=0]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
                                                    

                                                    And These:

                                                     

                                                    Jan 17, 2013 10:42:07 PM org.apache.catalina.core.AprLifecycleListener init
                                                    INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\\ColdFusion10\\cfusion\lib;C:\\ColdFusion10\\cfusion\jintegra\bin;C:\\ColdFusion10\\cfusion\jintegra\bin\international;C:\\ColdFusion10\\cfusion\lib\oosdk\classes\win
                                                    

                                                    You won't suprise me if you say these items may be related but I really don't know what those two other errors mean. I've tried googling them with no luck on what to do about them.

                                                     

                                                    Also looking at the exception log, usually around the same time I get an error stating that Coldfusion can't find something. Usually at the top of the stack the method its running is:

                                                     

                                                    at coldfusion.runtime.TemplateProxyFactory.getResolvedFile(TemplateProxyFactory.java:1344)

                                                     

                                                    At the time I am usually running something to get the path to create a component or something like that if I follow the trace down. This seems like it is the result of the main error that Coldfusion can't the path. Usually these exceptions are found just in the application.cfc or when the application is initializing. Just so everyone is aware, all our web files are on our D: drive not the system drive where Coldfusion is installed. It's not an external drive. This is a virtural server hosted by our IT department.

                                                     

                                                    All of the IIS6 Compatability components are there in the Role Services. Do I need to "turn them on?"

                                                    • 23. Re: ColdFusion 10 intermittent "service not available"
                                                      byrons1 Community Member

                                                      @Lee

                                                      FWIW I am having the exact same issue.  Installed CF10 on a fresh win 2k r2 box IIS7.5 running a Mura instance.  Developed the site over the last 6 months or so.  Ran into an issue with the site returning blank pages with a few errors.  I did see all of the ones you describe above.  Fiddled with the JVM settings to no avail.

                                                       

                                                      I hope adding my voice to the crowd gathering here helps get things fixed.

                                                      • 24. Re: ColdFusion 10 intermittent "service not available"
                                                        charlie@carehart.org Community Member

                                                        Lee and Byron, you may have what Byron says are the same issues, but your solutions may be different.

                                                         

                                                        First, Lee, you say you just applied update 7. Had you applied any CF10 updates prior to that? If so, had you re-run the web server configuration tool (to remove and re-add your connections)? That’s mandatory for updates 3-5.

                                                         

                                                        What many often don’t notice, though, is that if they may have “skipped” those updates and just applied a later one (like the latest, 7), then they STILL have to remember to rerun the web server config tool after that later update. The technote and update text (in the updates feature) do “recommend” it, but the challenge is that they don’t require it because technically it’s not required if you’re going from, say, update 6 to update 7. 

                                                         

                                                        But again, if you are either applying this as your first update, or had skipped one of those earlier ones, or had applied them but had NOT rerun the web server config tool, then you are currently running with the old connector. And THAT could be the cause of your problem.

                                                         

                                                        Look at the isapi_redirect.dll  file in each subdirectory of C:\ColdFusion10\config\wsconfig\ (meaning, C:\ColdFusion10\config\wsconfig\1, C:\ColdFusion10\config\wsconfig\2, and so on). Each of these represents a connection of IIS to an instance. You may have only one, you may have many such numbered subfolders. Look at the date of that file: if it’s not dated November 8 2012 (or later) then it’s not been updated. The connector in each subfolder must be updated.

                                                         

                                                        (And you run the tool for each instance from within that instance, such as C:\ColdFusion10\cfusion\instance1\bin\wsconfig.exe. Again, don’t forget to launch the tool with “run as administrator”, if on Windows Vista, 7 or 2008.)

                                                         

                                                        Finally, Byron, the reason I have proposed you may have the “same issue but different solution” is that you say you “Installed CF10 on a fresh win 2k r2”. If by that you mean you had not applied any updates at all, then you have 2 things to do: apply the latest update (7) and then run the web server config tool to reconfigure any connections you may have made during the installer.

                                                         

                                                        Let us know if this may have helped either of you (or other readers, seeing this in the future.)

                                                         

                                                         

                                                         

                                                        /charlie

                                                        • 25. Re: ColdFusion 10 intermittent "service not available"
                                                          Lee Bartelme Community Member

                                                          I think on our dev box, it made me do this when updating to hot fix 6. I don't remember specifically doing this on our prod box.  I tried searching for wsconfig.exe but didn't find it. Then I looked on adobes site on how to do this and the steps were more than confusing. I have no clue if I completed this step. I restarted the machine hoping it would complete this for me.  I'll check the date on the file and if it's older than what you said, look into doing that step again.  I swear my dev box prompted me to do this, but I don't remember the prod box doing so. If I remember right, the wsconfig tool just opens a single box witha single field right. I have no idea what I need to do at with the tool as there is little/no documentation on the subject. Can you point me in the right direction. I think on the dev box I just left whatever was in there and moved on.

                                                          • 26. Re: ColdFusion 10 intermittent "service not available"
                                                            charlie@carehart.org Community Member

                                                            No, Lee, it will not “complete it for you”. And I agree that if one looks at the CF docs (in the “Admin and Config” as well as “Install” guides) there is no real step-by-step guide to running the web server config tool. That is surprising.  It focuses only on how to “start” it, not how to “use it”.  I suppose some could argue, “it’s a gui, so it’s obvious”, but I would agree that someone not familiar with it would not know for sure how to work with it, and what all the buttons are about.

                                                             

                                                            (And those who think Lee is being whiny, consider also that someone who merely installed CF and chose to connect it to IIS at the time would NOT have previously seen this web server config tool. They only see it now, when the CF10 update calls for them to re-build the web server configuration. So this problem is perhaps more significant than it’s ever been for CFers without lots of CF admin/config experience.)

                                                             

                                                            But yes, Lee, you would need to run it (with that “run as administrator” option, if you see one when you right-click on it), and you would remove and then re-add whatever connections you see (taking note of what they were, if more than one, in terms of what “site” was connected to what “instance”, again if you run more than one instance of CF, as is supported in the Enterprise/Trial/Developer edition). A restart of CF won’t do it for you.

                                                             

                                                            Let us know if that helps.

                                                             

                                                             

                                                             

                                                            /charlie

                                                            • 27. Re: ColdFusion 10 intermittent "service not available"
                                                              Lee Bartelme Community Member

                                                              I updated the connectors last week, just had a crash 5 minuets ago.

                                                               

                                                              As a side note, I added a scheduled task on my development machine that would "ping" each of our applications every 20 minuets. If the server returned anything other than a 200 status, I have it send me an email. I had this script running for a for a few weeks before I updated my connector and the server never crashed. I stopped the scheduled task on the same day I updated the connector, wanting to see if it was my task that was keeping the server alive, or if this config would truly fix the issue.

                                                               

                                                              Oddly enough, when the application pools just stopped, I do not see the same getRealPathFromConn() error. This is in my error log at the time of the event:

                                                               

                                                              Feb 19, 2013 2:40:56 PM org.apache.catalina.core.StandardServer await

                                                              INFO: A valid shutdown command was received via the shutdown port. Stopping the Server instance.

                                                              Feb 19, 2013 2:40:56 PM org.apache.coyote.AbstractProtocol pause

                                                              INFO: Pausing ProtocolHandler ["ajp-bio-8012"]

                                                              Feb 19, 2013 2:40:56 PM org.apache.catalina.core.StandardService stopInternal

                                                              INFO: Stopping service Catalina

                                                              Feb 19, 2013 2:41:58 PM org.apache.catalina.core.AprLifecycleListener init

                                                              INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\\ColdFusion10\\cfusion\lib;C:\\ColdFusion10\\cfusion\jintegra\bin;C:\\ColdFusion10\\cf usion\jintegra\bin\international;C:\\ColdFusion10\\cfusion\lib\oosdk\classes\win

                                                              Feb 19, 2013 2:42:01 PM org.apache.coyote.AbstractProtocol init

                                                              INFO: Initializing ProtocolHandler ["ajp-bio-8012"]

                                                              Feb 19, 2013 2:42:01 PM org.apache.catalina.core.StandardService startInternal

                                                              INFO: Starting service Catalina

                                                              Feb 19, 2013 2:42:01 PM org.apache.catalina.core.StandardEngine startInternal

                                                              INFO: Starting Servlet Engine: Apache Tomcat/7.0.23

                                                              Feb 19, 2013 2:42:13 PM org.apache.catalina.core.ApplicationContext log

                                                              INFO: ColdFusionStartUpServlet: ColdFusion: Starting application services

                                                              Feb 19, 2013 2:42:13 PM org.apache.catalina.core.ApplicationContext log

                                                              INFO: ColdFusionStartUpServlet: ColdFusion: VM version = 20.4-b02

                                                              Feb 19, 2013 2:42:21 PM org.apache.catalina.core.ApplicationContext log

                                                              INFO: CFMxmlServlet: Macromedia Flex Build: 87315.134646

                                                              Feb 19, 2013 2:42:26 PM org.apache.coyote.AbstractProtocol start

                                                              INFO: Starting ProtocolHandler ["ajp-bio-8012"]

                                                              Feb 19, 2013 2:42:27 PM com.adobe.coldfusion.launcher.Launcher run

                                                              INFO: Server startup in 41013 ms

                                                               

                                                              I don't see an error. The first event there I believe (valid shutdown command) is when I hit restart on the windows machine. No errors. So what the heck? I had previously increased my IIS rapid fail to 50 and in event viewer of course there are 50 of them. Basically these two over and over again:

                                                               

                                                              • A process serving application pool 'domain.com' suffered a fatal communication error with the Windows Process Activation Service. The process id was '4924'. The data field contains the error number.
                                                              • A process serving application pool 'domain.com' terminated unexpectedly. The process id was '10336'. The process exit code was '0xff'.

                                                               

                                                              And at 2:36 Application pool 'domain.com' is being automatically disabled due to a series of failures in the process(es) serving that application pool.

                                                               

                                                              There are no coldfusion errors thrown today at all before 2:40, essentially when I restarted the server. Nothing in the exception log. I feel like all my CF update did was hide it from the log as the problem remains. I've turned my development scheduled task again in an attempt to keep the applcations running hoping there will be no issues. Any thoughts on this. I've been doing a lot of reading and it seems that the IIS 7.5, Win Server 2008, CF10 combo is entirely unstable.

                                                               

                                                              The isapi_redirect.dll located in Coldfusion10\config\wsconfig\1 is indeed dated 11/8/2012 1:18 PM.

                                                              • 28. Re: ColdFusion 10 intermittent "service not available"
                                                                Lee Bartelme Community Member

                                                                Chris,

                                                                 

                                                                Thank you for the note. We are have applied this fix, and so far so good.

                                                                 

                                                                We also had to update a setting in CF admin setting for: “Maximum number of simultaneous Template requests”. The default for CF 10 is set to 25. The default install of our local copy of CF 10 has this set to 25. Our box was set to 10, as it was upgraded from 8 and 9 and we think this was left over from CF 8 and 9.

                                                                 

                                                                However, according to Adobe, when the limit for Template requests is reached, the system is supposed to queue the rest. In our case it does not queue, it stops. So there is still some concern.

                                                                 

                                                                We also don’t see how a request is equal to site activity for 1 site. We thought that “worker processes” would be related to a number of different processes that happened when web templates run.

                                                                 

                                                                Ours is set to 0. Is that bad/incorrect. Does 0 in this case mean no limit?

                                                                • 29. Re: ColdFusion 10 intermittent "service not available"
                                                                  carl type3 Community Member

                                                                  Lee, Maybe some of the CF10\config\wsconfig\1\isapi_redirect.log around 2:40pm to 1 hour before would be interesting?

                                                                  Probably less so but still could be good to check anything of concern in that time frame for CF10\cfusion\logs\coldfusion-out.log?

                                                                  HTH, Carl.

                                                                  • 30. Re: ColdFusion 10 intermittent "service not available"
                                                                    Lee Bartelme Community Member

                                                                    So in the isapi_redirect.log there are alot of these at 14:34. And I mean A LOT, but they exist throughout:

                                                                     

                                                                    • [Tue Feb 19 14:34:02.404 2013] [9372:10556] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()
                                                                    • [Tue Feb 19 14:34:02.404 2013] [9372:10556] [error] init_jk::jk_isapi_plugin.c (2779): Initializing shm:(null) errno=-1. Load balancing workers will not function properly
                                                                    • [Tue Feb 19 14:34:02.420 2013] [9372:10556] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized
                                                                    • [Tue Feb 19 14:36:04.818 2013] [12000:11812] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250 (LESS OF THESE)

                                                                     

                                                                    After the restart sets of these:

                                                                     

                                                                    • [Tue Feb 19 14:41:00.206 2013] [9720:11712] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping
                                                                    • [Tue Feb 19 14:42:03.081 2013] [2944:2988] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()
                                                                    • [Tue Feb 19 14:42:28.805 2013] [2624:3044] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250

                                                                     

                                                                    There is nothing in the outlog that looks out of the ordinary or that looks like an error.

                                                                    • 31. Re: ColdFusion 10 intermittent "service not available"
                                                                      carl type3 Community Member

                                                                      Lee, case it helps here is an extraction of what ISAPI log looks like while working normally take from CF stop to load commencing.

                                                                       

                                                                      [Tue Feb 19 12:10:34.919 2013] [2596:1912] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping
                                                                      [Tue Feb 19 13:25:26.442 2013] [5888:5868] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()
                                                                      [Tue Feb 19 13:25:26.473 2013] [5888:5868] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized
                                                                      [Tue Feb 19 13:25:26.677 2013] [5888:5868] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250
                                                                      [Tue Feb 19 13:25:30.677 2013] [5888:3148] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 2 of max reuse connection 250 and total endpoint count 250
                                                                      [Tue Feb 19 13:25:30.692 2013] [5888:5868] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 3 of max reuse connection 250 and total endpoint count 250
                                                                      [Tue Feb 19 13:53:50.503 2013] [5888:3124] [info] ajp_send_request::jk_ajp_common.c (1658): (cfusion) all endpoints are disconnected, detected by connect check (3), cping (0), send (0)
                                                                      [Tue Feb 19 13:59:04.607 2013] [5888:2804] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 4 of max reuse connection 250 and total endpoint count 250
                                                                      [Tue Feb 19 13:59:05.060 2013] [5888:3124] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 5 of max reuse connection 250 and total endpoint count 250
                                                                      [Tue Feb 19 13:59:05.107 2013] [5888:4892] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 6 of max reuse connection 250 and total endpoint count 250

                                                                       

                                                                      > [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()
                                                                      I would think that odd if a stop did not proceed it. Did IIS WWW CF stop?
                                                                      > [error] init_jk::jk_isapi_plugin.c (2779): Initializing shm:(null) errno=-1. Load balancing workers will not function properly
                                                                      "Error" usually means there is a problem that needs to be resolved. I have not come across this one yet. Suggest follow that error detail perhaps will lead to solution.
                                                                      > [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized
                                                                      There we have part of a start-up process.
                                                                      > [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250
                                                                      Normal usage. Depending on load value for "reuse count is" will increment.

                                                                       

                                                                      While you can get more ISAPI log details via changing isapi_redirect.properties altering log_level=info to debug I would not recommend it in a production environment. If you know what the process is that leads up to crash and can simulate that process occurring then log_level=debug can be useful.

                                                                       

                                                                      Again, Carl.

                                                                      • 32. Re: ColdFusion 10 intermittent "service not available"
                                                                        carl type3 Community Member

                                                                        @Lee
                                                                        >INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path

                                                                         

                                                                        I don't think that will necessarily be causing a problem and you may not be using APR. Find information about APR here:


                                                                        http://tomcat.apache.org/native-doc/index.html

                                                                         

                                                                        You could download APR extract tcnative-1.dll (take care there is Win 32 and 64 bit) and copy that to CF10\cfusion\lib then restart CF10 Application Service. You will notice the log details change to EG:


                                                                        org.apache.catalina.core.AprLifecycleListener init
                                                                        INFO: Loaded APR based Apache Tomcat Native library
                                                                        org.apache.catalina.core.AprLifecycleListener init
                                                                        INFO: APR capabilities: IPv6 [true], sendfile [true], accept filters [false],
                                                                        random [true].
                                                                        org.apache.coyote.AbstractProtocol init
                                                                        INFO: Initializing ProtocolHandler ["ajp-apr-8012"]
                                                                        etc

                                                                        • 33. Re: ColdFusion 10 intermittent "service not available"
                                                                          Lee Bartelme Community Member

                                                                          After Googling " [error] init_jk::jk_isapi_plugin.c (2779): Initializing shm:(null) errno=-1. Load balancing workers will not function properly" I have come across others with the same issue dealing with the redirect.dll file. They speak of patching the file, but I don't think that is an option for me, as this dll should have to come from Adobe/CF correct?

                                                                           

                                                                          Should I be running my applicatin pools with Enable 32-bit set to true. They are set to false currently. This really seems like an Adobe CF/IIS issue doesn't it. What the hell is going on. I read in one thread they think this is happening durring applicatin recycle at some point or possibly due to load. My scheduled task that hits my applications every 20 minuets I think keeps all application/worker processes from shutting down, thus keeping the application pools alive, but my uneducation may prove it's all in my head.

                                                                           

                                                                          I have no idea if we are using APR or how to even tell. Should I just download the file and see what happenes. Is there some sort of drawback to doing so? Any harm that can be done? My thinking is this is a load issue as it NEVER happens on our development machine.

                                                                          • 34. Re: ColdFusion 10 intermittent "service not available"
                                                                            carl type3 Community Member

                                                                            As you say we can’t just go get the current Tomcat Connector (1.2.37) extract ISAPI DLL and use that. CF have given us a modified 1.2.32 Connector and I guess CF Dev team have good reasons for doing so.

                                                                             

                                                                            Having an error is a help, finding what the error really means can be a bit like following bread crumb trail. What to recommend I am not sure yet. Perhaps your workers.properties syntax would be interesting. I have been using JMX with jconsole to monitor load at the CF end of the Connector and found that helpful. Let me know if you want more details on that.

                                                                             

                                                                            Apply APR DDL if you want. I have on some CF10 environments with APR DLL applied but for now I think time spent on APR will distract from what the real problem is. Perhaps another day. I noticed it in the thread and so thought I would pass on what I know.

                                                                            • 35. Re: ColdFusion 10 intermittent "service not available"
                                                                              Lee Bartelme Community Member

                                                                              Thanks for your input. I'm a programmer assigned to also keep the server running if you know what I mean, so my skills as far as a Server Admin are pretty basic. I know what I am doing to get things done, but as far as real debugging/troubleshooting on the server side of things, I am lacking. I feel like my only solution is to wait for the next CF patch. I think my scheduled task keeps things running for now, and while not ideal, the server hasn't crashed while running the script yet.

                                                                              • 36. Re: ColdFusion 10 intermittent "service not available"
                                                                                carl type3 Community Member

                                                                                Lee - know where your coming from.

                                                                                 

                                                                                With the ISAPI log mention of "error" and "load balance" curious what is syntax of CF10\config\wsconfig\1\workers.properties EG:

                                                                                 

                                                                                worker.list=cfusion


                                                                                worker.cfusion.type=ajp13
                                                                                worker.cfusion.host=localhost
                                                                                worker.cfusion.port=8012
                                                                                worker.cfusion.max_reuse_connections=250

                                                                                 

                                                                                Read this informative blog, I know there is a lot there:

                                                                                 

                                                                                http://blogs.coldfusion.com/post.cfm/tuning-coldfusion-10-iis-connector-configuration

                                                                                 

                                                                                With your mention of running schedule task to keep things running that can point to timeout issues. You might do well to add time out syntax. IIS WWW and CF need restarting to apply changes. EG

                                                                                 

                                                                                workers.properties add -
                                                                                worker.cfusion.connection_pool_timeout = 60

                                                                                 

                                                                                server.xml CF10\cfusion\runtime\conf alter AJP section -
                                                                                <Connector port="8012" protocol="AJP/1.3" redirectPort="8445" tomcatAuthentication="false" connectionTimeout="60000" />

                                                                                 

                                                                                HTH again, Carl.

                                                                                • 37. Re: ColdFusion 10 intermittent "service not available"
                                                                                  Lee Bartelme Community Member

                                                                                  My workers.properties looks exactly as you have displayed. Should I try to corespond my connection_pool_timeout with my application timeout in my CF applications? Is 60 = 60 minuets? I'll read through your link in the meantime and hopefully it will answer my question of if I should have to edit the XML and the workers.properties or if it's an either or type of thing. Thanks again for your expertise.

                                                                                  • 38. Re: ColdFusion 10 intermittent "service not available"
                                                                                    carl type3 Community Member

                                                                                    workers.properties connection_pool_timeout = 60 is in seconds where as server.xml AJP connectionTimeout="60000" is in milliseconds. General recommendation is the time outs should match and references offer 60 seconds or 600 (10 minutes) for the ISAPI pool and AJP threads. I would not set to 60 minutes however as always do your own testing to know what value suits best. You are welcome - but I am no expert.

                                                                                    • 39. Re: ColdFusion 10 intermittent "service not available"
                                                                                      carl type3 Community Member

                                                                                      @Lee it has been a while. How is that going for you? Regards, Carl.

                                                                                      1 2 3 Previous Next