7 Replies Latest reply on Mar 4, 2007 10:31 AM by Newsgroup_User

    Accidental Session Leakage under load

    Girl_DIP
      The problem appears to only manifest itself when traffic peaks. Without giving figures for commercial reasons but we are talking significant amounts. To give you an idea, we are running a load-balanced cluster of 4 windows server at the moment and will have to upsize that shortly. We are not sharing sessions across the cluster and are using WLBS NLB Windows Load Balancing in single-instance mode with sticky sessions.

      The ID is the jsessionid, generated by CF.

      We have already locked-down all session variable access.

      As I see it there are only a few possibilities remaining:

      1) The load-balancing is not keeping a user on one server and the sessionid is not unique across servers.

      2) IIS is pushing cached content to the user (we put a cfheader fix for this possibility into application.cfm but it did not work!)

      3) CF is leaking under stress


      If you can think of any other possibilities then please let me know. All ideas appreciated.

      I am coming to the conclusion that the only way I can overcome this is to roll our own session solution using DB access and GUID/Encrypted GUID tokens that correspond to values in the user's record. Basically, double-checking every request and, if there is a discrepancy, pushing the user to the login screen and destroying the session.

      All thoughts more than welcome
        • 1. Re: Accidental Session Leakage under load
          Level 7
          You haven't REALLY described what it is that's actually occuring to make
          you think there's a problem.

          And what version of CF are you running?

          --
          Adam
          • 2. Accidental Session Leakage under load
            BKBK Adobe Community Professional & MVP
            Adam said:
            You haven't REALLY described what it is that's actually occuring to make
            you think there's a problem.

            A common mistake I, too, was once guilty of. Then I realized the troopers here usually don't read the topic's title or summary.

            Girl DIP said:
            double-checking every request and, if there is a discrepancy, pushing the user to the login screen and destroying the session.
            Better to locate and solve the problem than to lock down the session some more. Your users will soon get the feeling they're being hemmed in while the fair is going on outside.

            These problems are more often caused by code than by the underlying technology. I would therefore assume sticky sessions are working and keeping the user to one server. Check the login code. Also check how you attribute roles and profiles to users.


            • 3. Re: Accidental Session Leakage under load
              Girl_DIP Level 1
              We are using CFMX 7 Enterprise
              Windows 2003 Server Standard x64

              We have noticed other people have had this same problem.
              Any further thoughts?
              • 4. Re: Accidental Session Leakage under load
                BKBK Adobe Community Professional & MVP
                ... possibilities remaining:

                1) The load-balancing is not keeping a user on one server and the sessionid is not unique across servers.


                What does a typical jsessionid look like when the problem occurs?

                • 5. Re: Accidental Session Leakage under load
                  Grizzly9279 Level 1
                  Our production environment currently consists of a cluster of web servers (in our DMZ), that have JRun connectors installed that point to a multi-server CF cluster in a more protected subnet in our network.

                  We use Windows NLB for web servers (w/ IP affinity), but use ColdFusion's built-in clustering for the app servers (w/ J2EE session IDs, and sticky sessions).

                  We've not had any problems, and we do handle a fairly significant, and growing load of concurrent users.

                  I'm a little confused on if, or how your environment differs than ours. Are you using NLB for your APP servers? If so, I find that a little odd. Are you not using CF-clustering at all? Also, where are your web servers? Is each of your app servers also playing the web server role as well?

                  Using NLB for your app servers seems like a flawed approach to me. Can you clarify one some off these specifics a little more? There's must be an obvious explanation in here somewhere - I just feel like we're all going to need more info before we can get anywhere on this.

                  • 6. Re: Accidental Session Leakage under load
                    Level 7
                    > We have noticed other people have had this same problem.
                    > Any further thoughts?

                    Yes. My thoughts (still) are that I think you should describe the
                    manifestation of the problem more fully. You've not really given us much
                    to go on.

                    Just saying "it's not working" is not much help.

                    - does it happen that when a new visitor arrives at the site, they get
                    someone else's session instead of a new one?
                    - how is a user profile stored, and how is it fetched? What constitutes "a
                    different user"? It sounds like there is a login/authentication process
                    going on. Is that retrieving the correct user profile?
                    - does it happen half-way through a session?
                    - is their previous / correct session still there as well, which suggests
                    their session ID is changing some how?
                    - It's easy enough to find out if the sessions are hopping servers. Are
                    they?
                    - it's also easy enough to demonstrate whether IIS is returning cached
                    content. Is it?
                    - are you doing anything "clever" with session variables anywhere which
                    might not be working quite the way you think it is?
                    - what happens if you switch off J2EE sessions and just use CF ones
                    (temporarily)?
                    - what load balancing mechanism are you using?
                    - does it only happen with sessions on one of those servers, or all of
                    them?
                    - Does CF SEEM to be under load? What CPU usage is it hitting "at peak
                    times"? What RAM usage?
                    - what JVM RAM settings have you got in your JVM configs?
                    - what do your logs have to say for themselves?

                    Not related to your problem, but:
                    - why are you using Windows / IIS on a busy site?

                    --
                    Adam
                    • 7. Re: Accidental Session Leakage under load
                      Level 7
                      More things to think about:
                      - does it happen with ANY user at peak times, or only a subset of users?
                      - if they latter, what's the difference between users that DO switch
                      profiles, and those that don't?
                      - do the sessions get set wrong as soon as session management is being
                      enabled (ie: as soon as the site's <cfapplication> tag gets hit each
                      request, the user has the wrong session, or does it go wrong half-way
                      through a request?)
                      - Are you using <cfapplication> or application variables in
                      Application.cfc?

                      --
                      Adam