25 Replies Latest reply: Jun 11, 2013 7:28 PM by charlie@carehart.org RSS

    Help me figure this hack out.

    byblyk Community Member

      So as i was opening my presents with my family Christmas morning when my web server emailed me to let me know that a file had been created in my /CFIDE/ folder. This file was h.cfm and it was a nifty little tool used to scan files - copy files - dump SQL passwords - run commands - upload files ect....

       

      My setup is CF9.0.1 on win server 2008 with mySQL5.5 . The servers only purpose is to host a few websites for my company. I poured over my IIS logs and could not find any trace of a connection to the webserver while this was happening. I then started looking at the http.log file in coldfusion server and found that it contained 2 entries at the time of the attack. Both looked like this... note i've removed the IP of the server. This file is the file that was uploaded to my server.

       

      25-Dec-2012    6:54 AM    Information    jrpp-6969  Starting HTTP request {URL='http://IPAddress:80/CFIDE/h9.txt', method='get'} 

       

      There was nothing before this connection and nothing after it.  It looks like some how my server was told to make a call to here.  Nothing in any logs regarding the use of the h.cfm, no loggs in IIS showing suspicious page hit before and nothign suspicious after.

       

      Am i missing something??? this is driving me nuts! The other weird thing about this attack was that they didn't touch any websites they copied the ColdFusion9\updater_backup\ folder and a few other odd things off the server.

       

      Any insight into anything that went on would be helpful... even if it's just a guess that points me in the right direction.

       

      Thanks,

      JB.

        • 1. Re: Help me figure this hack out.
          charlie@carehart.org Community Member

          @JB, that's indeed a gnarly one. I found I'd been hit by it as well (also on Dec 25, in my case at 12:31am). Happy freakin' holidays. :-( So thanks very much for for sharing this news.

           

          I do have answers for you, and also info for others who may come across this.

           

          How it got there:

           

          As for your not having found any trace of it, I suspect it's because you're looking in the IIS logs for some one site--but it must then have gotten in on another site (either one you're not considering, or one for which you're not doing logging).

           

          I found a request for the following: /CFIDE/adminapi/administrator.cfc and then later a post to /CFIDE/Administrator/scheduler/scheduleedit.cfm. I won't share details beyond that, for now, to avoid this being exploited by others. But anyone looking to see how/when this h.cfm file got in there should look for that.

           

          As for the http.log entry you saw, that was the result of the scheduled task (cfprobe) they created. The task called that URL, and the result was saved to this h.cfm file. So the http call was not "sending" info, it was "getting" it.

           

          At a minimum, you (and anyone reading this who finds that h.cfm file) will want to delete it as wel as the scheduled task if it's there. (I could see in the IIS logs that the exploit tried to do that itself, to remove its tracks, but I found the task was still there, created that day.)

           

          Protecting against this attack (for other readers), Step 1:

           

          As for how others reading this can protect themselves against the attack, as the first link above shows, they're using an exploit in the Admin API. The bottom line protection for this is one that is often shared: lock down access to the CFIDE/administrator AND ALSO the CFIDE/adminapi directories. Whether you do that by IP address or by requiring other than anonymous authentication, you want to prevent unfettered access. You don't want to lock down the ENTIRE CFIDE, as there are other folders in there that do generally need to remain open for public access.

           

          This is all discussed further in the lockdown guides available for CF 8, 9, and 10. Just google for that, or see the recent discussion of it in the Adobe CF blog, which is linked to on the right side of this page.

           

          Now, someone may say, "so Charlie, you didn't have your CFIDE admin directories locked down"? Well, I did, at least for what I thought was the only publicly accessible site on my server. But it turns out I had a CFIDE in another site, the "default site".

           

          Let me share a tip/warning about that: many people may assume their "default site" (if not bound to any domain) is accessible then only via localhost. But note that most times, it's also bound to all IP addresses. So if a hacker knows your IP address (or any that your server will respond to), or if somehow some domain resolves to your server's IP address, then this default site will respond. I have now locked mine down (and again, there's more on this in the lockdown guides).

           

          Beware too that on CF10, if you use the web server config tool and set it to connect CF for "all sites", the tool adds a CFIDE virtual directory to every site. Again, someone may not think of that, and may not think to lock down the admin directories in each of those sites. Forewarned is forearmed.

           

          Update (added since initial posting): Finally, note that there's still another way that requests to the CFIDE/adminapi (and /administrator) directory might work, even if you have no such real or virtual directory defined for a site. See the discussion in the "part 2" blog entry that I refer to also in my additional comments below.

           

          Protecting against this attack (for other readers), Step 2:

           

          Of course, one should also be sure to implement all the security hotfixes available for their version of CF, as also linked to on the right side of this page and also discussed in the lockdown guide. I was running 9.0.2 without the latest two security fixes (though they're there now). I've not yet confirmed if I can recreate the exploit now Adobe has since confirmed that the exploit could happen even with current security hotfixes in place (but again, having locked down the adminapi directory, that's the more important point of prevention, for now.)

           

          Determining what was done:

           

          As for determing what, if any, the hacker may have done, one should look in their web server logs (if you've have enabled them, I hope) to find any references to that h.cfm file. You may see some that get a 404 (made before they succeeded in getting it in place). But then once it's in place you may then see other calls to it, perhaps indicating a fuseaction in the query string showing what it was doing. Not all the things that can be done with the tool result in a fuseaction, though. It may just do a post back to itself, as I find for the things on the page that are form submissions (the command feature, the search feature, etc.) But the tool always calls itself, so look for that h.cfm file in your logs. (Of course, at some point the hacker may realize we're on to them and change the file to a new name.)

           

          Determining what can be done:

           

          While any who get hit with this should delete the h.cfm file if it's there, you may want to move it somewhere else (in a protected directory) to explore more about what it can or can't do. I don't mean so much to "discover how it can be used" (the code in the file is open, not encoded or precompiled), but to find out what if anything could have been done with it on your own server, depending on what protections you may have in place.

           

          For instance, I found that on my server, the feature it tries to use to obtain datasources did not work. The drop-down of DSNs was never populated. That may be because of other security fixes already in place (9.0.2 does contain all those that existed for 9.0.1 as of the time 9.0.2 was created in May).

           

          In fact, I found that the hacker only hit the h.cfm page one time, and not again (even over the 3 days since). Perhaps they felt that since they couldn't get that info, they weren't as interested in trying to use it further. That said, many other features of the hack tool did work, so again it's very important that people remove it, and also do what they can to prevent it hitting them (lock down those adminapi and administrator directories!)

           

          Conclusion:

           

          I'll continue to research and will decide how best to share my findings (whether here, in a blog entry, or just with Adobe). Of course, there's always a balancing act in sharing too much info publicly about an exploit. It can give info to others who might also leverage the exploit, and of course it also lets the bad guys of this one know we're on to them. But that was true as soon as you posted this, if they may find it.

           

          Of course, I do thank you very much for sharing it. (And I realize someone from Adobe may even remove my entry here, if they feel that it shares too much info already, though I've tried to be careful to share only info on mitigating/analyzing the problem, not leveraging its exploits.)

          • 2. Re: Help me figure this hack out.
            rupesh_kumar Adobe Employee

            @JB,

            This indeed sounds very serious and we would need your help to investigate this. Could you please get in touch with psirt@adobe.com with all the details that you have?

             

            - Rupesh

            Adobe ColdFusion Team

            • 3. Re: Help me figure this hack out.
              Adam Cameron. Community Member

              Good stuff Charlie.

               

              TBH, I think in CF11 Adobe need to decouple the scripts that stuff like CFFORM etc need from any of the administrator or admin API files, so this necessity to keep anything in the CFIDE directory externally accessible is removed.

               

              Also, I think when CF is being configured on IIS or Apache web servers (ie: during install), the admin stuff should not be made available on those web sites. They should only be available via the internal site, browseable via localhost.

               

              The default install of CF should not facilitate this sort of access.

               

              And the CFAdmin should never be exposed on a public-facing website, yet it is by default.

               

              :-|

               

              --
              Adam

              • 4. Re: Help me figure this hack out.
                byblyk Community Member

                I'm in contact with Adobe right now, the logs and anything else relevant are going over to them today when i get into work. 

                 

                As for Adobe's  security, I’m appalled that this day-in-age the default security would be so lax, we do pay for a product after all and i expect a certain level of professionalism that comes with that.  I thought i had the CFIDE folder locked down but apparently not.  I've taken the approprate steps now to ensure it's secured from prying eyes.

                • 5. Re: Help me figure this hack out.
                  Adam Cameron. Community Member

                  Esp when one considers they have a person with a title of "Security Tzar", and the potential security risks associated with leaving /CFIDE open like that by default was brought to their attention prior to the release of CF10. They too-hard-basketed it.

                   

                  --

                  Adam

                  • 6. Re: Help me figure this hack out.
                    byblyk Community Member

                    @Charlie

                     

                    Thank you for taking the time to post this.  It was driving me up the wall.  When i read your post i rushed home to check the server logs and sure enough it was there.  The reason i didn't see it the first time is because while the enitial hack happened at 6:54am the log enteries are not until 11:30am.  I"m going to have to investigate this further but thats another issue. 

                    • 7. Re: Help me figure this hack out.
                      charlie@carehart.org Community Member

                      +1 on those recommendations, Adam. We’ve been fighting this too long, and given the ever tightening security focus, this is one for which it seems finally time to bite the bullet.

                       

                       

                       

                      /charlie

                      • 8. Re: Help me figure this hack out.
                        lcarr Community Member

                        Add me to the list of folks hit by this one.  In addition to the h.cfm file, there was also an i.cfm file with only the words "Connection Failure" in it.  We're in the process of going through the IIS logs to see when/if either of the files have been accessed. According to the date stamp, h.cfm was created this morning 1/2/13 at 10:12am. The other file, i.cfm, was created today at 10:11am.  So if the datestamp is to be trusted, the files were on the server less than 4 hours when I saw Charlie's blog post pointing here on Twitter.

                        • 9. Re: Help me figure this hack out.
                          Russ Michaels Community Member

                          I would suggest anyone who has not locked down their CFIDE or has been affected by this hack reads the following article.

                          http://www.michaels.me.uk/post.cfm/securing-your-coldfusionmx-installation-on-windows

                          It is pretty old now as it was written for CF6, but still applies.

                          • 10. Re: Help me figure this hack out.
                            Creayt

                            Interestingly ( maybe to Adobe ), the requests for h.cfm all seemed to come from different IPs.

                             

                            Here is a list of ones that I saw on a VM we had ( luckily our important production stuff was locked down enough that it wasn't vulnerable).

                             

                            87.236.194.191

                            173.254.216.66

                            141.101.237.49

                            64.250.127.236

                            93.182.129.86

                            37.59.123.176

                            37.59.162.218

                            108.174.195.211

                            77.109.139.87

                            199.48.147.39

                            List goes on.

                             

                            Agent: python-requests/0.14.2+CPython/2.7.3+Linux/3.2.0-32-generic

                             

                            Also noticed they requested this a few  times: /CFIDE/help.cfm

                             

                            And this ( scary ): /CFIDE/scripts/ajax/FCKeditor/editor/filemanager/connectors/cfm/connector.cfm Command=GetFoldersAndFiles&Type=File&CurrentFolder=/

                            • 11. Re: Help me figure this hack out.
                              robert_d_redford Community Member

                              I also have a handful for help.cfm.

                               

                              Where did these files come from? Did the hackers place them there?

                               

                              Blocking anonymous access to adminapi and administrator will help lock down CFIDE without breaking anything, though?         

                              • 12. Re: Help me figure this hack out.
                                Russ Michaels Community Member

                                see the link I posted previously for the solution to lock down the CFIDE

                                • 13. Re: Help me figure this hack out.
                                  Denard Springle Community Member

                                  Thank you Charlie!

                                  • 14. Re: Help me figure this hack out.
                                    robert_d_redford Community Member

                                    My bad -- I missed that, but I did follow the directions to locking down the directories. Thanks for the link.

                                    • 15. Re: Help me figure this hack out.
                                      charlie@carehart.org Community Member

                                      Yes, Blood, locking those down is the bottom line (as I noted in my first reply). And Russ does offer one approach. That said (and as he admits), his entry is "pretty old" and there's some info that I think should be added based on changes since CF8 (he wrote it for CF7). I just made a comment there with some additional observations, which are awaiting moderation at the moment.

                                       

                                      Anyway, I've had many people ask for a pointer to more on locking down these folders. There's no one good document I've found yet (even the lockdown guide coverage of it is sparse). There've been various recommended approaches over the years. Some involve making multiple copies of the CFIDE, some involve locking things down in the web server (whose steps will vary for IIS and Apache), and there are still others. Each has its pros and cons, and no one (I've seen) has reviewed them all with observations of such pros/cons.

                                       

                                      For instance, a frequent suggestion is that one could make a copy of the CFIDE folder (to be modified), first leaving the "full" one pointed to in any site(s) that are meant to have the Admin, while then modifying the copied folder to remove the /administrator and /adminapi directories, with that copied CFIDE being pointed to by other sites as a virtual directory (or worse, just multiple copies of this modified CFIDE placed in each site).

                                       

                                      My concern with doing that (even making just one modified copy) is that with recent hotfixes, cumulative hotfixes, and security hotfixes calling for one to extract updated CFIDE files from a zip, it's then CRITICAL that whoever does that knows to extract them into BOTH (or ALL) of the copies. Sometimes it's different people doing things, or the one person may forget that they made the copies. I've seen this be the source of lots of pain (and blogged about it: CF911: Are you finding CF (or CF Admin) busted after applying a hotfix? Three possible reasons

                                       

                                      As for protecting these /CFIDE/adminapi and /CFIDE/administrator directories using other means, there have been various blog entries on the topic (with different approaches), including:

                                      http://www.morgankelsey.com/post/how-to-lock-down-cfide-in-iis

                                      http://www.aaronwest.net/blog/index.cfm/2010/10/4/Blocking-ColdFusion-Administrator-in-Apa che

                                      http://www.petefreitag.com/item/750.cfm

                                      http://www.petefreitag.com/item/774.cfm

                                      http://www.clarke.ca/post.cfm/coldfusion-administrator-lockdown

                                      http://www.raymondcamden.com/index.cfm/2007/5/11/Ask-a-Jedi-Password-protecting-CFIDE

                                      http://www.talkingtree.com/blog/index.cfm/2005/7/20/SecureAdmin

                                      (and Russ's:

                                      http://www.michaels.me.uk/post.cfm/securing-your-coldfusionmx-installation-on-windows)

                                       

                                      Hope that helps someone.

                                      • 16. Re: Help me figure this hack out.
                                        robert_d_redford Community Member

                                        Chariie,

                                         

                                        Thanks for those links -- also very helpful to see some other ways of addressing this.

                                         

                                        From a comment on Ray's site from Charlie's links above:

                                        We basically put CFIDE under the default website and make it available only from localhost and the internal network IP. Unless I'm mistaken people will never be able to hit it unless they're on the VPN.

                                        Has anyone tried this? Does it work?

                                        • 17. Re: Help me figure this hack out.
                                          charlie@carehart.org Community Member

                                          Blood, well, no, you can't just do that (if by that he meant "the entire CFIDE"), because the CFIDE/scripts (and some other subdirectories) do need to be public, for use by HTML created by various CFML tags (such as the new CF8+ ajax ones, and of course the older cfform ones, and others).

                                           

                                          As I said in my first entry, and repeated in my last reply to you, what you need to do is lock down the /CFIDE/adminapi and /CFIDE/administrator directories, whether you lock it down via IP address or via requiring additional web server authentication. If you do that, then you'll be protected from this exploit where the /CFIDE/adminapi folder was wide open ("pants down").    

                                          • 18. Re: Help me figure this hack out.
                                            byblyk Community Member

                                            When I was running IIS 6 this is the method i used and it stopped all the shannagans with the CFIDE folder.

                                            http://www.morgankelsey.com/post/how-to-lock-down-cfide-in-iis

                                             

                                            Now that i'm running IIS 7.5 i've decided to try the new request filtering so i've added this to my web.config file.  Not the most elegant way of doing it.  I'd much perfer a deny all requests to CFIDE then an allow CFIDE/scripts but that didn't seem to work. 

                                             

                                            <security>
                                                     <requestFiltering>
                                                        <!-- block /CFIDE -->
                                                        <denyUrlSequences>
                                                           <add sequence="/CFIDE/administrator"/>
                                                           <add sequence="/CFIDE/adminapi"/>
                                                           <add sequence="/CFIDE/AIR"/>
                                                           <add sequence="/CFIDE/appdeployment"/>
                                                           <add sequence="/CFIDE/componentutils"/>
                                                           <add sequence="/CFIDE/debug"/>
                                                           <add sequence="/CFIDE/orm"/>
                                                           <add sequence="/CFIDE/portlets"/>
                                                           <add sequence="/CFIDE/probe.cfm"/>
                                                           <add sequence="/CFIDE/services"/>
                                                           <add sequence="/CFIDE/wizards"/>
                                                           <add sequence="/CFIDE/ServerManager"/>
                                                           <add sequence="/CFIDE/scripts"/>    
                                                        </denyUrlSequences>

                                                       
                                                        <!-- hide configuration dir -->
                                                        <hiddenSegments applyToWebDAV="true">
                                                           <add segment="configuration" />
                                                        </hiddenSegments>
                                                        <!-- limit post size to 10mb, query string to 256 chars, url to 1024 chars -->
                                                        <requestLimits maxQueryString="256" maxUrl="1024" maxAllowedContentLength="102400000" />
                                                        <!-- only allow GET,POST verbs -->
                                                        <verbs allowUnlisted="false" applyToWebDAV="true">
                                                           <add verb="GET" allowed="true" />
                                                           <add verb="POST" allowed="true" />
                                                        </verbs>
                                                     </requestFiltering>
                                                    </security>

                                            • 19. Re: Help me figure this hack out.
                                              robert_d_redford Community Member

                                              Hi Charlie,

                                               

                                              We did lock down those directories and the admin is also no longer accessible unless you are on our VPN. Hopefully our pants are back up now!

                                               

                                              Thanks again,

                                               

                                              Rob

                                              • 20. Re: Help me figure this hack out.
                                                ascott67

                                                I do the same thing for SQL Injection as well, not an overall solution but it helps as well as the coding techniques.

                                                • 21. Re: Help me figure this hack out.
                                                  charlie@carehart.org Community Member

                                                  I just want to point out that since offering my initial reply above the other day, I have today posted two blog entries with still more detail to help readers better understand, protect themselves from, and understand the implications/more deeply investigate if hacked by this particular exploit:

                                                   

                                                  http://www.carehart.org/blog/client/index.cfm/2013/1/2/serious_security_threat

                                                   

                                                  http://www.carehart.org/blog/client/index.cfm/2013/1/2/Part2_serious_security_threat

                                                   

                                                  Hope that's helpful.

                                                  • 22. Re: Help me figure this hack out.
                                                    plarts Community Member

                                                    I have all of these problem in my CF production server, I am very afraid.

                                                     

                                                    I am not enough knowlegeable in security and all you said for prevention.

                                                     

                                                    Then , I cannot try all those things.

                                                     

                                                    The only thing I will do is :

                                                    put a <cfabort> in the application.cfm

                                                    of CFIDE and CFIDE/administration

                                                     

                                                    Thanks to tell me if this is silly solution ?

                                                    What is the risk ?

                                                     

                                                    Thanks again,

                                                    • 23. Re: Help me figure this hack out.
                                                      charlie@carehart.org Community Member

                                                      That would not solve the problem completely, nor do I think it would be a wise attempt in any case.

                                                       

                                                      First, while some of the hack was about running code in the CF admin, other parts of the code are about running code via a web request to the AdminAPI directory instead. Second, some of the hacks involve running code in still other directories (like the SCRIPTS directory).

                                                       

                                                      What you really need to do, at minimum, is apply the security hotfix(es) from Adobe. You should be able to do that rather easily if you simply follow the instructions from the Adobe hotfix technotes.

                                                       

                                                      (And also, even if you did want to “block the ability to use the CF Admin” this way, for whatever other benefit that may offer you, do you mean you would take that change out when ready to run the admin intentionally? Otherwise you’d never be able to run your admin. Finally, even if you did want to do that, note that you would have to replace the application.cfm in the admin with a new one having this abort, because you can’t edit the one that comes with it: it’s encoded, so not human-readable or -editable.)

                                                       

                                                      There are many other things you can do, some easier than others, and some more fundamental and universally recommended than others (some of which I discussed in a part 3 follow-up to the two earlier entries mentioned earlier: http://www.carehart.org/blog/client/index.cfm/2013/1/15/Part3_serious_security_threat).

                                                       

                                                      But seriously, it really is important to protect your environment, so if you really feel you can’t do any of them at all yourself, hiring someone to help (like Pete @ foundeo, or myself, even for as little as an hour) would be highly recommended, for your sake and the sake of all your customers and anyone else on your server.

                                                       

                                                      /charlie

                                                      • 24. Re: Help me figure this hack out.
                                                        plarts Community Member

                                                        Thanks for your answer,

                                                        Could you make me proposal to do that update security job ?

                                                        Because, I am too afraid to make it myself under production.
                                                        I suppose that a Remote Desk Top access would be enough to do that ?
                                                        I am running CF9.0 under Windows 2008 server R2.
                                                        I did not apply the Windows update SP 1.
                                                        How much time do you think it could take ?

                                                         

                                                        Thanks , Pierre.

                                                        • 25. Re: Help me figure this hack out.
                                                          charlie@carehart.org Community Member

                                                          So sorry for missing this, Pierre. I could certainly help, and do appreciate why some people feel it's too much to try on their own. If you're still interested, please drop me a note directly at charlie@carehart.org.