9 Replies Latest reply on Apr 28, 2011 11:19 AM by Drew747

    DataGrid high rate insert rows

    matias2681

      Hi everyone,

       

      I'm working in a project where we have a high rate messages (like 2-3k per second) . We use Java on server side, BlazeDS and Flex.

      We have a problem on flex side, the messages are insert on a DataGrid where we just show the last 1k messages, after the 100k messages passed through to the application, the datagrid start to refresh slower than before.

       

      Do you know how can i solve this problem?

       

      I executed the profiler and i saw that we don't have memory leaks

       

      Thanks in advance.

        • 1. Re: DataGrid high rate insert rows
          Level 2

          which version of Flex ? So if i understand correctly you never have more than 1000 in your grid ?  Are you clearing the contents of the grid before the next batch of 1000 or strictly overwriting them in place ?

           

          you are not regenerating your grid are you, that is, you are re-using the same instance ?

           

          probably recycling and garbage collection issue.

          • 2. Re: DataGrid high rate insert rows
            matias2681 Level 1

            Hi dpierce747,

             

            We are using Flex 4.

            Exactly as you said, we always show at maximun 1000 message, that are injected in realtime at a rate of 2-3k message per second.

            When the message arrive we overwrite the list, that is the provider, with a set operation.

            Definitely we don't regenerate the grid, always use the same instance of datagrid.

            The evidence that i have from the profiler is that there aren't memory leaks and the memory from the flash plugin is stable while the message arrive.

            • 3. Re: DataGrid high rate insert rows
              Level 2

              can you describe the complexities of the columns (like how many and approx byte width of each) and are your getting 2 to 3 k rows per second initially and then after 100k of 1k increments what is the performance like. about how long does it take before it slows and how slow?

               

              what does task manager look like at the beginning and at the end when you sort Processes by CPU. Wondering if it is just the browser overwhelmed with the housekeeping necessary for the flash pluggin. Does it start out mostly idle with iexplorer.exe at a low amt and then iexplorer spikes ?

              • 4. Re: DataGrid high rate insert rows
                matias2681 Level 1

                There are 7 columns and the messages are around 200 bytes each, this means that the 200 bytes are split in the 7 columns.

                Those messages are encapsulated in an object Message inside of which there are MessageFields that have two properties, tag and value.

                Initially i sent 2.5k message per second but i never tried to increase the rate while the message were been delivered, when i resolve this issue i am going to try that.

                At the begin (with 2.5k per sec) the refresh rate is quite good (almost realtime), i mean that the datagrid refreshes the sequence number of the messages really fast, each second i saw the sequence number increase in 2.5k.

                After the first minute the datagrid starts to collapse and it refreshes every few seconds, like 5 seconds for example, and once its refreshed the sequence number increases in 5 * 2.5k, so the message are coming at the rate.

                 

                Regarding the use of cpu i underestimated that point, i was focus on the memory usage but i've showed that flash plugin recently it's between the first 3rd process and sometime took the 1st place.

                How i can reduce the use of cpu?

                • 5. Re: DataGrid high rate insert rows
                  matias2681 Level 1

                  is it a good idea to elimiate the collection provider and add the rows one by one?

                  • 6. Re: DataGrid high rate insert rows
                    Level 2

                    sorry if i sound dense but i am picking up on the phrase "but i never tried to increase the rate "

                     

                    i don't know anything about your architecture but are you cramming data back in some forced rate ?  are you force streaming ? what exactly are you doing  !!

                     

                    that poor flash player sounds overwhelmed.

                     

                    Date: Thu, 28 Apr 2011 10:05:12 -0600

                    From: forums@adobe.com

                    To: drewpierce@live.com

                    Subject: DataGrid high rate insert rows

                     

                    There are 7 columns and the messages are around 200 bytes each, this means that the 200 bytes are split in the 7 columns.

                    Those messages are encapsulated in an object Message inside of which there are MessageFields that have two properties, tag and value.

                    Initially i sent 2.5k message per second but i never tried to increase the rate while the message were been delivered, when i resolve this issue i am going to try that.

                    At the begin (with 2.5k per sec) the refresh rate is quite good (almost realtime), i mean that the datagrid refreshes the sequence number of the messages really fast, each second i saw the sequence number increase in 2.5k.

                    After the first minute the datagrid starts to collapse and it refreshes every few seconds, like 5 seconds for example, and once its refreshed the sequence number increases in 5 * 2.5k, so the message are coming at the rate.

                     

                    Regarding the use of cpu i underestimated that point, i was focus on the memory usage but i've showed that flash plugin recently it's between the first 3rd process and sometime took the 1st place.

                    How i can reduce the use of cpu?

                     

                    >

                    • 7. Re: DataGrid high rate insert rows
                      Level 2

                      are you sending data back to a client async or something from a client that is not ready to receive it ?

                       

                      are you forcing it upon them or are they requesting the next chunk ?

                      • 8. Re: DataGrid high rate insert rows
                        matias2681 Level 1

                        We have developed an application (for test propouse) that send a number of message per seconds specified by the user, so we are forcing the streaming at a fixed rate.

                        The conexion between server-client is by push, so when the message arrive to the cliente we add the object to the collection provider.

                        • 9. Re: DataGrid high rate insert rows
                          Level 2

                          More info required on technique. use vbr like fms server

                           

                          Date: Thu, 28 Apr 2011 11:54:31 -0600

                          From: forums@adobe.com

                          To: drewpierce@live.com

                          Subject: Flex DataGrid high rate insert rows

                           

                          We have developed an application (for test propouse) that send a number of message per seconds specified by the user, so we are forcing the streaming at a fixed rate.

                          The conexion between server-client is by push, so when the message arrive to the cliente we add the object to the collection provider.

                          >