5 Replies Latest reply on Jul 21, 2013 7:24 AM by Chandra gupta

    Load Testing AEM 5.6.1

    Chandra gupta Level 1

      We got  4 web server and 4 AEM 5.6.1 publish instances.  each web server is  individually connected with publish.

      HP QC is being used for load testing. Publish has 6 static pages juts like geomatrixx.

      If test is run for around 50 or 60 users concurrent users ,response time is less than 2 seconds for an hour but if it ran for 250 users for same period response time goes up to 130 seconds. Average response time is 35 seconds.

       

      I have verified following things

       

      1. Memory usage on CQ is always less than 500 MB from system console and there is hardly any hit on CQ because pages are cached in dispatcher.

      2. Apache and dispatcher logs not showing any issue and dispatcher log level is set 1.

      3. We have verified from network and no issue has been identified yet.

       

      Same 6 pages as static html has been verified for 250 users on same hardware setup using QA and response time was  less than 2 seconds now those pages converted into CQ but showing these issue.

       

      Please share any thoughts. Thanks Chandra

        • 1. Re: Load Testing AEM 5.6.1
          JayanKandathil Level 4

          Chandra:

           

          The behaviour you are seeing tells me that it is time to tune your HTTP Server.  Depending on the MPM module you are using, you will have to increase certain parameters in httpd.conf

           

          - Jayan

           

          Jayan Kandathil

          Adobe Managed Services

          • 2. Re: Load Testing AEM 5.6.1
            Chandra gupta Level 1

            Okay but i don't see any request is taking longer than few ms in apache loogs. I just checked HP QC logs and find few request has page not found and page's dom content not loaded so QC wait goes for timeout and wait untill 200  some time 1800 seconds that's why we see so much long responses

            • 3. Re: Load Testing AEM 5.6.1
              Chandra gupta Level 1

              Here is error log  from apache error.

              [Thu Jul 18 00:58:00 2013] [notice] mpmstats: rdy 1199 bsy 1 rd 1 wr 0 ka 0 log 0 dns 0 cls 0

              [Thu Jul 18 01:28:02 2013] [notice] mpmstats: rdy 1197 bsy 3 rd 0 wr 0 ka 3 log 0 dns 0 cls 0

              [Thu Jul 18 06:48:11 2013] [notice] mpmstats: rdy 1197 bsy 3 rd 0 wr 0 ka 3 log 0 dns 0 cls 0

              [Thu Jul 18 08:38:14 2013] [notice] mpmstats: rdy 1197 bsy 3 rd 0 wr 0 ka 3 log 0 dns 0 cls 0

              [Thu Jul 18 09:08:15 2013] [notice] mpmstats: rdy 1197 bsy 3 rd 0 wr 0 ka 3 log 0 dns 0 cls 0

              [Thu Jul 18 10:48:18 2013] [notice] mpmstats: rdy 1198 bsy 2 rd 0 wr 1 ka 1 log 0 dns 0 cls 0

              [Thu Jul 18 10:48:18 2013] [notice] mpmstats: bsy: 1 in disp_apache2.c

              [Thu Jul 18 10:58:19 2013] [notice] mpmstats: rdy 1135 bsy 65 rd 0 wr 0 ka 64 log 0 dns 0 cls 1

              [Thu Jul 18 11:08:19 2013] [notice] mpmstats: rdy 1196 bsy 4 rd 0 wr 0 ka 4 log 0 dns 0 cls 0

              [Thu Jul 18 11:18:19 2013] [notice] mpmstats: rdy 1155 bsy 45 rd 0 wr 0 ka 45 log 0 dns 0 cls 0

              [Thu Jul 18 11:28:19 2013] [notice] mpmstats: rdy 1126 bsy 74 rd 0 wr 0 ka 74 log 0 dns 0 cls 0

              [Thu Jul 18 11:38:20 2013] [notice] mpmstats: rdy 1099 bsy 101 rd 0 wr 0 ka 101 log 0 dns 0 cls 0

              [Thu Jul 18 11:48:20 2013] [notice] mpmstats: rdy 1142 bsy 58 rd 0 wr 0 ka 58 log 0 dns 0 cls 0

              [Thu Jul 18 11:58:20 2013] [notice] mpmstats: rdy 1198 bsy 2 rd 0 wr 0 ka 2 log 0 dns 0 cls 0

              • 4. Re: Load Testing AEM 5.6.1
                Jörg Hoh Adobe Employee

                Chandra,

                 

                check the request.log on your CQ side for response times. Then use the "recent request" tab in the Felix console to find out where the time is spent when answering the request. Leveraging this feature you can drill down to which individual servlet or JSP takes how much time. To go even deeper you need a profiler (or you work with threaddumps ...)

                 

                Anyhow, you should start the regular performance analysis process now.

                 

                Jörg

                • 5. Re: Load Testing AEM 5.6.1
                  Chandra gupta Level 1

                  Thanks For reply... But as i said request is not reaching to Even apache if you see the apache log MPM thread shows more than 1100 worker thread are in ready state.

                  Any way I found load generators HP QC itself was prevented sending request to Apache by network firewall and now i am ok with the understanding.