SEVERE: The web application [/] created a ThreadLocal

[Phil_Cruz]

Looking in coldfusion-error.logs I'm seeing a bunch of entries like

SEVERE: The web application [/] created a ThreadLocal with key of type [coldfusion.util.DateUtils$1] (value [coldfusion.util.DateUtils$1@6c59782f]) and a value of type [java.util.GregorianCalendar] (value [java.util.GregorianCalendar[time=1350663029000,areFieldsSet=true,areAllFieldsSet=true,lenient=false,zone=sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]],firstDayOfWeek=1,minimalDaysInFirstWeek=1,ERA=1,YEAR=2012,MONTH=9,WEEK_OF_YEAR=42,WEEK_OF_MONTH=3,DAY_OF_MONTH=19,DAY_OF_YEAR=293,DAY_OF_WEEK=6,DAY_OF_WEEK_IN_MONTH=3,AM_PM=0,HOUR=9,HOUR_OF_DAY=9,MINUTE=10,SECOND=29,MILLISECOND=0,ZONE_OFFSET=-28800000,DST_OFFSET=3600000]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

Oct 19, 2012 10:36:08 AM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks

SEVERE: The web application [/] created a ThreadLocal with key of type [coldfusion.util.DateUtils$1] (value [coldfusion.util.DateUtils$1@6c59782f]) and a value of type [java.util.GregorianCalendar] (value [java.util.GregorianCalendar[time=1350667603000,areFieldsSet=true,areAllFieldsSet=true,lenient=false,zone=sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]],firstDayOfWeek=1,minimalDaysInFirstWeek=1,ERA=1,YEAR=2012,MONTH=9,WEEK_OF_YEAR=42,WEEK_OF_MONTH=3,DAY_OF_MONTH=19,DAY_OF_YEAR=293,DAY_OF_WEEK=6,DAY_OF_WEEK_IN_MONTH=3,AM_PM=0,HOUR=10,HOUR_OF_DAY=10,MINUTE=26,SECOND=43,MILLISECOND=0,ZONE_OFFSET=-28800000,DST_OFFSET=3600000]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

Oct 19, 2012 10:36:08 AM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks

What could be causing that and how to resolve?

Thanks,

Phil

[conanbarb]

I'm getting the same output in my ColdFusion-error.log files.  It seems to be preceded by the following lines

INFO: Stopping service Catalina

Nov 2, 2012 10:24:21 AM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks

Maybe it's just me, but anytime I see SEVERE in an error log I have some concern.

I currently have ColdFusion 10 Update 4 installed and I was getting the same errors with Update 2

Anybody have some input on this?

[Charlie Arehart]

[I'm updating my first reply here, from 2012 (when cf10 was released) to point readers to a much later reply I offered in 2015, currently at the end of this thread (now on the second page of them here). There I point out how this "problem" seems not really at all a "severe" one, despite the message. It's simply a message that appears at shutdown, whose number will depend on the number of cf sessions at the time. The messages don't CAUSE any hang, nor are they implying the result of any. 

Some may prefer to read the subsequent messages below, but even the offered workaround to change the jvm has not prevented the problem, which remains even in cf2021, as I write this update to my first comment from 2012.

HTH. The rest of my comment here was my first, naive, one as we scrambled in 2012 to understand this then-new message which started appearing with cf10, released that year. ]

Since you mention having applied updater 4, have you re-run the web server configuration? That’s a requirement for the CF10 updaters (mentioned on the updater page, but easily overlooked).

it may not make any difference, but I wanted throw it out in case it might.

Similarly, be aware that sometimes when you run the web server config tool, it may seem to “work” when it has not. I’ve found that it’s necessary to “run it as admin”, which means doing start>adobe>coldfusion 10 and then right-clicking on “web server onfig tool” to choose “run as administrator”.

Hope that may help.

/charlie

[conanbarb]

I did run the wsconfig tool as instructed, but I did not "run as administrator" as you suggested.  I'll give that a shot and let you know if anything changes.

[conanbarb]

I used the "run as administrator" on the wsconfig tool and I'm still getting the SEVERE errors.  They seems to show up in the ColdFusion-error.log file after the "ColdFusion 10 Application Server" service is restarted.

Any ideas?

Thanks,

Christian

[Charlie Arehart]

OK. I’d next recommend you check the logs for the CF10 updater 4 you say you applied, to make sure it was really applied and without any errors. Since it was supposed to fix these sort of problems, we’d really want to make sure it was applied before concluding that it did not. The logs for the udpaters are in a hf-updates dir, within the CF10 instance where they were applied (such as C:\ColdFusion10\cfusion\).

Speaking of that, are you by any chance running multiple instances? And is your web server set to connect to one of the instances other than the default cfusion instance? If so, can you confirm as well that you applied the updater to THAT instance? Besides looking in the CF Admin for that instance, again look in the logs for the update, such as C:\ColdFusion10\instancex\hf-updates\.

/charlie

[conanbarb]

I've installed Update 4 on both my Dev and Test servers.  They are both single instances and they are also both behaving the same.  The SEVERE errors exist before and after Update 4.

I looked at the log files for Update 4 and everything says "Status: SUCCESSFUL" so it looks like it was installed succesfully on both servers.  I can email the log file directly to you if you'd like.

My servers are 64bit Windows Server 2008 R2 on VM ESX 4.0 for your reference.

Thanks,

Christian

[Charlie Arehart]

Well, emailing the logs to me won’t help much. And especially if they say that all operations were “successful”.

But along those lines, are you confirming that it’s saying that for all actions within the log? Worth double-checking.

If that’s not the issue, and you are still having issues, I’d recommend you raise your concern on one of the blog entries (at blogs.coldfusion.com), to get the attention of the engineers. They are not required to respond here (or there, technically) but you just may get their attention their as far more people read the blog than the forums, I’d suspect.

Hope you resolve the problem eventually.

/charlie

[conanbarb]

That is correct, all of the actions have "Status: SUCCESSFUL" in the Update 4 log file.

I'll check out the blogs.  I appreciate your input, Charlie.

Thanks,

Christian

[iManagementMark]

Were you able to find a resolution for this?  I too am experiencing this error on Update 2 and now Update 4.

Version	10,0,4,283281
Tomcat Version	7.0.23.0

SEVERE: The web application [/] created a ThreadLocal with key of type [coldfusion.util.DateUtils$1] (value [coldfusion.util.DateUtils$1@202d2c30]) and a value of type [java.util.GregorianCalendar] (value [java.util.GregorianCalendar[time=1352746379829,areFieldsSet=true,areAllFieldsSet=true,lenient=false,zone=sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]],firstDayOfWeek=1,minimalDaysInFirstWeek=1,ERA=1,YEAR=2012,MONTH=10,WEEK_OF_YEAR=46,WEEK_OF_MONTH=3,DAY_OF_MONTH=12,DAY_OF_YEAR=317,DAY_OF_WEEK=2,DAY_OF_WEEK_IN_MONTH=2,AM_PM=0,HOUR=10,HOUR_OF_DAY=10,MINUTE=52,SECOND=59,MILLISECOND=829,ZONE_OFFSET=-28800000,DST_OFFSET=0]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

I checked out http://blogs.coldfusion.com/ but don't see any info.

Thanks,

Mark

[Leftbower]

and I have the exact same issue with Updater 6 installed. Has there been any resolution to this?

I am havng issues witrh getting the console to work in CFB after updating to CF10 on all 3 of my dev machines (this thread: http://forums.adobe.com/thread/1113052) and was looking through the CF logs for possible reasons and stumbled across this error.

So not saying they have any relation, but now that makes 2 items that are troubling after moving from CF9 to 10...

[RobertAOwen]

I am also seeing this error.  Our CF10 box is a fresh install not an upgrade (not that that makes any difference) with Update 6 installed.  I do have multiple instances installed on this box and have veriffied that update 6 was successful on all.  I have re-run the configuration tool for each instance as "Administrator" and we still see the issue.  Has anyone made any progress with this??

SEVERE: The web application [/] created a ThreadLocal with key of type [coldfusion.util.DateUtils$1] (value [coldfusion.util.DateUtils$1@1b99ba1]) and a value of type [java.util.GregorianCalendar] (value [java.util.GregorianCalendar[time=1355777649150,areFieldsSet=true,areAllFieldsSet=true,lenient=false,zone=sun.util.calendar.ZoneInfo[id="America/New_York",offset=-18000000,dstSavings=3600000,useDaylight=true,transitions=235,lastRule=java.util.SimpleTimeZone[id=America/New_York,offset=-18000000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]],firstDayOfWeek=1,minimalDaysInFirstWeek=1,ERA=1,YEAR=2012,MONTH=11,WEEK_OF_YEAR=51,WEEK_OF_MONTH=4,DAY_OF_MONTH=17,DAY_OF_YEAR=352,DAY_OF_WEEK=2,DAY_OF_WEEK_IN_MONTH=3,AM_PM=1,HOUR=3,HOUR_OF_DAY=15,MINUTE=54,SECOND=9,MILLISECOND=150,ZONE_OFFSET=-18000000,DST_OFFSET=0]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

[BKBK]

This definitely points to a severe problem. But it is not with your code. This occurs at the level of the Java Virtual Machine, and a ColdFusion library is the cause. A datetime class in the Java package coldfusion.util behaves in a way that can cause a memory leak.

It is significant that this occurs during shutdown. As the error message says, one or more ThreadLocal date objects that had been created couldn't be garbage-collected during shut-down. Such runaway rogue threads may retain references that survive a restart of a web application. This can eventually lead to a memory leak.

The following line from the error message tells us why we shouldn't panic: "Threads are going to be renewed over time to try and avoid a probable memory leak". However, the ColdFusion team should look into this while the going is still good.

[Corey_Stafford]

Any updates on this I am getting the same still in CHF 8 on ubuntu 12.04. Server is crashing about once every 24 hr, unknown yet if this is related to the cause.

Thanks,

~ Ben

[michaelmuller]

I am having the same problem. 

When I run wsconfig.exe I have an empty list under "Configured Web Servers."  When I click Add... I get the following:

Note that the OK button is greyed out. If I choose another web server and then select IIS again, I do get the OK button. When I click Ok I get the message "An unknown error ocurred."

I'm confused by all this, because my server is functioning, so ... why does this tool think I'm not connected?

[Anit_Kumar]

Hi,

Did you try running the tool in Admin mode? If that doesn't work, please refer to http://help.adobe.com/en_US/ColdFusion/10.0/Admin/WSc3ff6d0ea77859461172e0811cbf364104-7fd9.html for running the wsconfig tool via command line.

Regards,

Anit Kumar

[loftx]

Hi there,

We've also been experiencing the same errors in our logs after the clock change to BST here in the UK.

I have been unable to find any other information relating to this issue online apart from this thread so have posted a bug to https://bugbase.adobe.com/index.cfm?event=bug&id=3540118

I will update this with any further information I find.

Kind regards,

Tom

[lukester85]

Hi Tom,

I have added a note and a vote to your bug post.

The note i have added is below. If anyone has any suggestions for a workaround please do post them.

I have tried running the webserver config as administrator but this has not resolved the situation.
Changing your clock to Pacific time seems to stop the error, but then the server is on the wrong time.

Kind regards
Luke

Note added to Toms bug report:
I am also seeing this behavior, this is a new install updated to CF10 update 9. I have re-ran the webserver config tool as admin as per suggestions here: http://forums.adobe.com/thread/1085240 but to no effect.

The server clock is set to time zone (UTC) Dublin, Edinburgh, Lisbon, London.

If i change the clock to (UTC -0800) Pacific Time (US & Canada) i do not get the error.

The error in the logs is: Apr 11, 2013 4:02:18 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks

SEVERE: The web application [/] created a ThreadLocal with key of type [coldfusion.util.DateUtils$1] (value [coldfusion.util.DateUtils$1@6bbdcdd5]) and a value of type [java.util.GregorianCalendar] (value [java.util.GregorianCalendar[time=1365690925920,areFieldsSet=true,areAllFieldsSet=true,lenient=false,zone=sun.util.calendar.ZoneInfo[id="Europe/London",offset=0,dstSavings=3600000,useDaylight=true,transitions=242,lastRule=java.util.SimpleTimeZone[id=Europe/London,offset=0,dstSavings=3600000,useDaylight=true,startYear=0,startMode=2,startMonth=2,startDay=-1,startDayOfWeek=1,startTime=3600000,startTimeMode=2,endMode=2,endMonth=9,endDay=-1,endDayOfWeek=1,endTime=3600000,endTimeMode=2]],firstDayOfWeek=2,minimalDaysInFirstWeek=4,ERA=1,YEAR=2013,MONTH=3,WEEK_OF_YEAR=15,WEEK_OF_MONTH=2,DAY_OF_MONTH=11,DAY_OF_YEAR=101,DAY_OF_WEEK=5,DAY_OF_WEEK_IN_MONTH=2,AM_PM=1,HOUR=3,HOUR_OF_DAY=15,MINUTE=35,SECOND=25,MILLISECOND=920,ZONE_OFFSET=0,DST_OFFSET=3600000]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak

[lukester85]

For anyone who has this issue with time and CF10 - are you using the version of Java that comes bundled with CF?

I was and i found that installing Java as a stand-alone (ie to c:\program files) then modifying the JVM.config file to point java home to the new location has solved the error.

Download Java from here: http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html

install it

navigate to C:\ColdFusion10\cfusion\bin\jvm.config

backup
open in notepad

change "java.home=C:\\ColdFusion10\\jre" to the new location - this was "java.home=C:\\Program Files\\Java\\jdk1.7.0_17\\jre" for me

save

restart CF

[jkanzen]

We had four SEVERE errors on the pre-shipped JRE wirh CF10 64 bit. We installed the latest Sun JRE and upon restarting the server this has reduced the errors to just one. However, on a second restart we are back to 4 SEVERE errors again.

We also noticed that our memory heap is large when starting the server (we use FusionReactor) and unless a garbage dump is done, this memory usage stays high. The garbage dump seems to release a massive amount of memory too (254MB!).

Anyone got any ideas to solve this one?

[Charlie Arehart]

Interesting observation, wombat. So to be clear, are you referring to JVM 6 or 7? CF 10 originally “pre-shipped” (as of May 2012) with Java 6, but a new installer (for Windows only) in March of 2013 causes it to come with Java 1.7. (And a hotfixe released at the same time also allowed anyone running CF10 or CF9 to change to Java 1.7.

So it may be helpful for readers to know which you are referring to (for both what you found “pre-shipped” and what you changed to).

Also, while I don’t mean to cause embarrassment or pick a fight, I do want to offer another clarification of a common misconception: doing a GC (or a “garbage dump”, as you referred to it) is never necessary to solve a memory problem. Even if you find the heap to be high (and yes, CF can indeed hold the heap high for a few minutes after startup), there’s no reason for you to do a GC to try to “make that go away”. The fact is that if and when the JVM gets memory pressure (because of requests running) and more heap is needed, it is THEN when the JVM would do a GC--on its own.

Now, if it does (or you try to do) a GC, but it can’t collect much, then that’s a different problem. Something is holding memory, and if that is not relieved, then you may well get an outofmemory error.

But bottom line: doing a GC is only something that helps you see how much memory can be garbage collected. It’s not something you need to do. The JVM will do it itself, if it senses a need to.

Hope that’s helpful.

/charlie

[cf_null_user]

We are experiencing this problem as well. It sounds like lukester85 has solved it by having CF use an install of Java outside of the internal Coldfusion Java install.  Has anyone tried this who has been eperiencing this problem?

[tribule]

This solved it for us too.

[Stephane_Chauvin]

Hi,

I tried to install a JRE outside of Coldfusion Java install, but I still have the SEVERE log entries...

[Feb]

Hi Stephane/All,

Have you solved this issue?

I have intalled a newer Java version, but it did not help.

Thanks!