Array bottleneck...need assistance

[DongerNeedFood]

OK, I have a bit of code that is BOGGING down my app. I need some ideas for smoothing it out. Basically, I need to be able to reference an array by 3 elements (id, qstn_id, zone_id). In a query later, I sum up the data and assign it to the proper position in the array (ie zone1[6552][124747][16543] = 2). When you create an array, does it create all of the elements below the element I have created? For the example above, does CF have to create positions for 1-6551 for the first element, 1-124746 for the second, etc? Would it help any to reference the position in the original reference list rather than the actual IDs? Ex. instead of zone1[6552][124747][16543], it would be zone1[1][1][1].

I'm not married to the idea of the array. My other idea was to assign dynamic variables to the zones, but I don't think it would help.

Any help/ideas will be appreciated.

[DongerNeedFood]

Also, I just ran this little bit of code, along with the query to assign the values to the array. It ran in about 8 seconds, but I watched the Memory Usage for jrun.exe spike from 61MB to 385MB in that time. Do arrays really have this affect? Or are my array limits just too big? It's also been about 5 minutes and my mem usage has only gone down to 135MB. I'm using MX7 on my test server, so I'm the only one on it.

[CFMunster]

You are creating a giant empty array, which is why performance is so slow. Keep in mind that CFMX code turns into Java code, and your CF array becomes Vector object in Java.

Have you considered using a structure instead? Then you could use your IDs as keys and avoid the overhead of creating a big empty array for which Java has to allocate and deallocate memory.

[DongerNeedFood · Accepted solution]

I tried a couple of solutions:

First, I made the array positions be the list index values instead of the actual ID.

So instead of: cfset xrayzones1[ i ][ q ][ z ] = 0

I used: cfset xrayzones1[listfind(id_lst,i)][listfind(qstn_list,q)][listfind(zone_list,z)] = 0

This solution improved processing time dramatically and decreased mem usage. But I wanted to see if structures would improve it more.

My second attempt involved using structures to declare my zones. Ultimately, I went with structures because it produced the fastest processing time. My processing time went from ~8seconds to 0.35seconds. Here's the code:

[Newsgroup_User]

On Wed, 10 May 2006 15:21:14 +0000 (UTC), DongerNeedFood wrote:

> I tried a couple of solutions:
>
> First, I made the array positions be the list index values instead of the
> actual ID.
>
> So instead of: cfset xrayzones1 = 0
>
> I used: cfset xrayzones1 = 0
>
> This solution improved processing time dramatically and decreased mem usage.
> But I wanted to see if structures would improve it more.
>
> My second attempt involved using structures to declare my zones. Ultimately,
> I went with structures because it produced the fastest processing time. My
> processing time went from ~8seconds to 0.35seconds. Here's the code:
>
> <cfset theid = "i#i#q#q#z#z#">
> <cfset a = structinsert(zones1,theid,0)>

You could more closely replicate your original array-based code with a
couple of quick mods.

<cfset zones = structNew()>
<cfset zones[1] = structNew()>
<cfloop list="#id_lst#" index="i">
<cfset zones[1] = structNew()>
<cfloop list="#qstn_list#" index="q">
<cfset zones[1]

quote:

---

= structNew()>
<cfloop list="#zone_list#" index="z">
<cfset zones[1]

quote:

---

= 0>
</cfloop>
</cfloop>
</cfloop>

<cfset zones[2] = duplicate(zones[1])>
<cfset zones[3] = duplicate(zones[1])>
<cfset zones[4] = duplicate(zones[1])>

Obviously if the real assignment logic is more complex than creating four
identical structs, all holding some keys with value 0, then this
duplicate() bit won't work, but you get my point, I'm sure.

--
Adam

[James74]

Could you get by with a 2 dimensional array and you only need to declare it once:

<cfset YourArray = ArrayNew(2)>

<CFSET YourArray[index][1] = id_list>
<CFSET YourArray[index][2] = qstn_list>
<CFSET YourArray[index][3] = zone_list>

Additionally, You may want to consider researching structures for this