5 Replies Latest reply on May 10, 2006 9:37 AM by Newsgroup_User

    Array bottleneck...need assistance

    DongerNeedFood
      OK, I have a bit of code that is BOGGING down my app. I need some ideas for smoothing it out. Basically, I need to be able to reference an array by 3 elements (id, qstn_id, zone_id). In a query later, I sum up the data and assign it to the proper position in the array (ie zone1[6552][124747][16543] = 2). When you create an array, does it create all of the elements below the element I have created? For the example above, does CF have to create positions for 1-6551 for the first element, 1-124746 for the second, etc? Would it help any to reference the position in the original reference list rather than the actual IDs? Ex. instead of zone1[6552][124747][16543], it would be zone1[1][1][1].

      I'm not married to the idea of the array. My other idea was to assign dynamic variables to the zones, but I don't think it would help.

      Any help/ideas will be appreciated.
        • 1. Re: Array bottleneck...need assistance
          DongerNeedFood Level 1
          Also, I just ran this little bit of code, along with the query to assign the values to the array. It ran in about 8 seconds, but I watched the Memory Usage for jrun.exe spike from 61MB to 385MB in that time. Do arrays really have this affect? Or are my array limits just too big? It's also been about 5 minutes and my mem usage has only gone down to 135MB. I'm using MX7 on my test server, so I'm the only one on it.
          • 2. Re: Array bottleneck...need assistance
            CFMunster
            You are creating a giant empty array, which is why performance is so slow. Keep in mind that CFMX code turns into Java code, and your CF array becomes Vector object in Java.

            Have you considered using a structure instead? Then you could use your IDs as keys and avoid the overhead of creating a big empty array for which Java has to allocate and deallocate memory.
            • 3. Array bottleneck...need assistance
              James74 Level 1
              Could you get by with a 2 dimensional array and you only need to declare it once:

              <cfset YourArray = ArrayNew(2)>

              <CFSET YourArray[index][1] = id_list>
              <CFSET YourArray[index][2] = qstn_list>
              <CFSET YourArray[index][3] = zone_list>

              Additionally, You may want to consider researching structures for this
              • 4. Re: Array bottleneck...need assistance
                DongerNeedFood Level 1
                I tried a couple of solutions:

                First, I made the array positions be the list index values instead of the actual ID.

                So instead of: cfset xrayzones1[ i ][ q ][ z ] = 0

                I used: cfset xrayzones1[listfind(id_lst,i)][listfind(qstn_list,q)][listfind(zone_list,z)] = 0

                This solution improved processing time dramatically and decreased mem usage. But I wanted to see if structures would improve it more.

                My second attempt involved using structures to declare my zones. Ultimately, I went with structures because it produced the fastest processing time. My processing time went from ~8seconds to 0.35seconds. Here's the code:
                • 5. Re: Array bottleneck...need assistance
                  Level 7
                  On Wed, 10 May 2006 15:21:14 +0000 (UTC), DongerNeedFood wrote:

                  > I tried a couple of solutions:
                  >
                  > First, I made the array positions be the list index values instead of the
                  > actual ID.
                  >
                  > So instead of: cfset xrayzones1 = 0
                  >
                  > I used: cfset xrayzones1 = 0
                  >
                  > This solution improved processing time dramatically and decreased mem usage.
                  > But I wanted to see if structures would improve it more.
                  >
                  > My second attempt involved using structures to declare my zones. Ultimately,
                  > I went with structures because it produced the fastest processing time. My
                  > processing time went from ~8seconds to 0.35seconds. Here's the code:
                  >
                  > <cfset theid = "i#i#q#q#z#z#">
                  > <cfset a = structinsert(zones1,theid,0)>

                  You could more closely replicate your original array-based code with a
                  couple of quick mods.

                  <cfset zones = structNew()>
                  <cfset zones[1] = structNew()>
                  <cfloop list="#id_lst#" index="i">
                  <cfset zones[1] = structNew()>
                  <cfloop list="#qstn_list#" index="q">
                  <cfset zones[1]
                  quote:

                  = structNew()>
                  <cfloop list="#zone_list#" index="z">
                  <cfset zones[1]
                  quote:

                  [z] = 0>
                  </cfloop>
                  </cfloop>
                  </cfloop>

                  <cfset zones[2] = duplicate(zones[1])>
                  <cfset zones[3] = duplicate(zones[1])>
                  <cfset zones[4] = duplicate(zones[1])>

                  Obviously if the real assignment logic is more complex than creating four
                  identical structs, all holding some keys with value 0, then this
                  duplicate() bit won't work, but you get my point, I'm sure.

                  --
                  Adam