8 Replies Latest reply on Dec 23, 2015 1:48 PM by Carl Von Stetten

    cfthread in cfloop, all variables passed in on the thread, intermittent java.util.concurrentmodificationexception.  cfloop is a query loop and each thread calls cfquery.  How do I find the error?

    Lance-b Level 1

      Code

       

      Below is the script, the queries in the thread only use the variables on the cfthread call. The process works about 80% of the time and then I receive Java.util.concurrentmodificationexception on one of the threads, occasionally more than one.  Using CF 9,0,0,251028, but have tested with 9,0,2,282541 with the same issue.

       

      Any hekp is appreciated

       

      <cfquery name = "variables.Dts" datasource="#mydsn#">

      select 

      to_char(next_day(to_date('#selDate#','mm/dd/yyyy')-56,'sat')+(level-1)*7-2,'mm/dd/yyyy') as thu,

      to_char(next_day(to_date('#selDate#','mm/dd/yyyy')-56,'sat')+(level-1)*7-1,'mm/dd/yyyy') as fri,

      to_char(next_day(to_date('#selDate#','mm/dd/yyyy')-56,'sat')+(level-1)*7,'mm/dd/yyyy') as sat

      from dual

      connect by  level <  9

      </cfquery>

       

       

      <!--- test --->

      <cfset qtr = 0>

      <cfset tname = ''>

       

      <cfloop query="variables.dts" >

      <cfset qtr += 1>

       

      <cfthread name="t#qtr#" action="run" sat2='#variables.dts.sat#' thu2='#variables.dts.thu#' selarea2='#selArea#' selDistrict2='#seldistrict#' qtr2='#qtr#' dsn="#mydsn#" >

       

       

      <cfquery name="variables.m#qtr2#" datasource="#dsn#" >

      ...

       

      </cfquery>

       

      </cfthread>

      <cfset tname &= 't#qtr#' & ','>

      </cfloop>

       

       

      <cfloop query="variables.dts" >

      <cfset qtr += 1>

       

      <cfthread name="t#qtr#" action="run" sat1='#variables.dts.sat#' fri1='#variables.dts.fri#' selarea1='#selArea#' selDistrict1='#seldistrict#' qtr1='#qtr#' dsn="#mydsn#">

       

       

      <cfquery name="variables.m#qtr1#" datasource="#dsn#" >

      ...

      </cfquery>

       

      </cfthread>

      <cfset tname &= 't#qtr#' & ','>

      </cfloop>

       

       

      <cfset tname = left(tname,len(tname)-1)>

      <cfthread action="join" name="#tname#" />

        • 1. Re: cfthread in cfloop, all variables passed in on the thread, intermittent java.util.concurrentmodificationexception.  cfloop is a query loop and each thread calls cfquery.  How do I find the error?
          Carl Von Stetten Adobe Community Professional & MVP

          I moved this to the Advanced Techniques subforum because many of the more experienced ColdFusion community members hang out there.  If you don't get an answer, you might try joining the CFML Slack channel, as you can get access to more real-time community assistance.  I'll even put a link there to this thread to see if I can draw some attention.

           

          -Carl V. - Moderator

          • 2. Re: cfthread in cfloop, all variables passed in on the thread, intermittent java.util.concurrentmodificationexception.  cfloop is a query loop and each thread calls cfquery.  How do I find the error?
            BKBK Adobe Community Professional & MVP

            What is your motivation for using threads? I cannot see any, yet the threads add a lot of complexity to the code.

             

            The database is an independent system. It should therefore be treated as a black-box which may have its own concurrency strategy, beyond Coldfusion's control. Before you know it, there are worms everywhere.

             

            Aside from that, if you replace <cfset tname &= 't#qtr#' & ','> with <cfset tname = listAppend(tname, 't#qtr#')> you will save yourself one line of code.

            • 3. Re: cfthread in cfloop, all variables passed in on the thread, intermittent java.util.concurrentmodificationexception.  cfloop is a query loop and each thread calls cfquery.  How do I find the error?
              Lance-b Level 1

              Thank you for the reply

               

               

               

              The threads are being used to speed up the retrieval of the data.  It works well, I can shave minutes off the  total query time.  Each query returns the data of a single partition and is combined  in a  dbquery. It provides in a week analysis of selected facilities almost instantly.  It is exactly what is needed, if I can resolve the concurrentmodificationexception. Thank you for the listAppend suggestion, much more readable.

              • 4. Re: cfthread in cfloop, all variables passed in on the thread, intermittent java.util.concurrentmodificationexception.  cfloop is a query loop and each thread calls cfquery.  How do I find the error?
                BKBK Adobe Community Professional & MVP

                You have omitted some code (marked by ellipsis), so it is difficult to say.  In any case, the hint from the concurrentmodificationexception is that a thread th6, say, attempted to modify a variable variables.somevar while thread th7was still busy with it.

                 

                As your threads are asynchronous, execution may already move to the second loop while threads of the first loop are still busy. Is that consistent with your business logic?

                • 5. Re: cfthread in cfloop, all variables passed in on the thread, intermittent java.util.concurrentmodificationexception.  cfloop is a query loop and each thread calls cfquery.  How do I find the error?
                  Lance-b Level 1

                  I agree with your assessment and appreciate your time, I tried to  eliminate any access by the thread to local variables. I have added the complete code (below), if you have time to consider

                   

                   

                   

                   

                   

                  The below code works most of the time , but occasionally a thread ( in this example t7) will reurn this error

                   

                   

                   

                  T7          

                   

                  struct

                   

                  ELAPSEDTIME    0

                   

                  ERROR 

                   

                  struct

                   

                  Message             

                   

                  StackTrace          java.util.ConcurrentModificationException at java.util.WeakHashMap$HashIterator.nextEntry(WeakHashMap.java:762) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:801) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:799) at java.util.HashMap.putAll(HashMap.java:523) at coldfusion.monitor.memory.MemoryTracker.getAllPrimitiveMap(MemoryTracker.java:92) at coldfusion.monitor.memory.MemoryTracker.getEstimatedMemory(MemoryTracker.java:117) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:100) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:30) at coldfusion.monitor.memory.MemoryTrackerProxy.onAddObject(MemoryTrackerProxy.java:277) at coldfusion.runtime.NeoPageContext.setVariableScope(NeoPageContext.java:1546) at coldfusion.runtime.NeoPageContext.SymTab_initializeForPage(NeoPageContext.java:963) at coldfusion.runtime.NeoPageContext.initializeWith(NeoPageContext.java:293) at coldfusion.thread.Task.run(Task.java:131) at coldfusion.scheduling.ThreadPool.run(ThreadPool.java:201) at coldfusion.scheduling.WorkerThread.run(WorkerThread.java:71)

                   

                  TagContext        

                   

                  array

                   

                  Type      java.util.ConcurrentModificationException

                   

                  NAME   T7

                   

                  OUTPUT              

                   

                  PRIORITY             NORMAL

                   

                  STARTTIME         {ts '2015-12-20 19:06:13'}

                   

                  STATUS                TERMINATED

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                  *************

                   

                  ,'mm/dd/yyyy') start_the_clock_date,

                   

                          sum(piece_cnt) AS BLUE_tot, sum(INCL_SVC_VAR_MINUS4_CNTINCL_SVC_VAR_MINUS3_CNTINCL_SVC_VAR_MINUS2_CNT) as blue,

                   

                          0 AS ORANGE_tot, 0 as orange,hyb_svc_std_desc 

                   

                          from V_mstr a

                   

                          inner join BI_MSTR_SVC_STD b

                   

                          on b.ORGN_ENTERED = 'N'

                   

                          and b.ML_CL_CODE= a.ML_CL_CODE

                   

                          and b.SVC_STD = a.SVC_STD

                   

                          INNER JOIN

                   

                          V_facility c

                   

                          ON c.fac_SEQ_ID = ORGN_FAC_SEQ_ID

                   

                          inner join

                   

                         (select distinct site_name as area_name,area,district_name,district

                   

                         from v_hier) ad

                   

                          on ad.district = c.district        

                   

                          where a.start_the_clock_date = to_date(

                   

                   

                   

                   

                   

                   

                   

                   

                   

                  *********************

                   

                   

                   

                   

                   

                   

                   

                  William R Lance

                   

                  425.408.0855

                   

                  425.482.0375 - Cell

                   

                  Pacific Time Zone

                   

                   

                   

                  The below code works most of the time , but occasionally a thread ( in this example t7) will reurn this error

                   

                   

                   

                  T7          

                   

                  struct

                   

                  ELAPSEDTIME    0

                   

                  ERROR 

                   

                  struct

                   

                  Message             

                   

                  StackTrace          java.util.ConcurrentModificationException at java.util.WeakHashMap$HashIterator.nextEntry(WeakHashMap.java:762) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:801) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:799) at java.util.HashMap.putAll(HashMap.java:523) at coldfusion.monitor.memory.MemoryTracker.getAllPrimitiveMap(MemoryTracker.java:92) at coldfusion.monitor.memory.MemoryTracker.getEstimatedMemory(MemoryTracker.java:117) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:100) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:30) at coldfusion.monitor.memory.MemoryTrackerProxy.onAddObject(MemoryTrackerProxy.java:277) at coldfusion.runtime.NeoPageContext.setVariableScope(NeoPageContext.java:1546) at coldfusion.runtime.NeoPageContext.SymTab_initializeForPage(NeoPageContext.java:963) at coldfusion.runtime.NeoPageContext.initializeWith(NeoPageContext.java:293) at coldfusion.thread.Task.run(Task.java:131) at coldfusion.scheduling.ThreadPool.run(ThreadPool.java:201) at coldfusion.scheduling.WorkerThread.run(WorkerThread.java:71)

                   

                  TagContext        

                   

                  array

                   

                  Type      java.util.ConcurrentModificationException

                   

                  NAME   T7

                   

                  OUTPUT              

                   

                  PRIORITY             NORMAL

                   

                  STARTTIME         {ts '2015-12-20 19:06:13'}

                   

                  STATUS                TERMINATED

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                  *************

                   

                  ,'mm/dd/yyyy') start_the_clock_date,

                   

                          sum(piece_cnt) AS BLUE_tot, sum(INCL_SVC_VAR_MINUS4_CNTINCL_SVC_VAR_MINUS3_CNTINCL_SVC_VAR_MINUS2_CNT) as blue,

                   

                          0 AS ORANGE_tot, 0 as orange,hyb_svc_std_desc 

                   

                          from V_mstr a

                   

                          inner join v_SVC_STD b

                   

                          on b.ORGN_ENTERED = 'N'

                   

                          and b.ML_CL_CODE= a.ML_CL_CODE

                   

                          and b.SVC_STD = a.SVC_STD

                   

                          INNER JOIN

                   

                          V_facility c

                   

                          ON c.fac_SEQ_ID = ORGN_FAC_SEQ_ID

                   

                          inner join

                   

                         (select distinct site_name as area_name,area,district_name,district

                   

                         from v_hier) ad

                   

                          on ad.district = c.district        

                   

                          where a.start_the_clock_date = to_date(

                   

                   

                   

                   

                   

                   

                   

                   

                   

                   

                  • 7. Re: cfthread in cfloop, all variables passed in on the thread, intermittent java.util.concurrentmodificationexception.  cfloop is a query loop and each thread calls cfquery.  How do I find the error?
                    Lance-b Level 1

                    I agree with your assessment and appreciate your time, I tried to  eliminate any access by the thread to local variables. I have added the complete code (below), if you have time to consider

                     

                     

                     

                     

                     

                    The below code works most of the time , but occasionally a thread ( in this example t7) will reurn this error

                     

                     

                     

                    T7          

                     

                    struct

                     

                    ELAPSEDTIME    0

                     

                    ERROR 

                     

                    struct

                     

                    Message             

                     

                    StackTrace          java.util.ConcurrentModificationException at java.util.WeakHashMap$HashIterator.nextEntry(WeakHashMap.java:762) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:801) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:799) at java.util.HashMap.putAll(HashMap.java:523) at coldfusion.monitor.memory.MemoryTracker.getAllPrimitiveMap(MemoryTracker.java:92) at coldfusion.monitor.memory.MemoryTracker.getEstimatedMemory(MemoryTracker.java:117) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:100) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:30) at coldfusion.monitor.memory.MemoryTrackerProxy.onAddObject(MemoryTrackerProxy.java:277) at coldfusion.runtime.NeoPageContext.setVariableScope(NeoPageContext.java:1546) at coldfusion.runtime.NeoPageContext.SymTab_initializeForPage(NeoPageContext.java:963) at coldfusion.runtime.NeoPageContext.initializeWith(NeoPageContext.java:293) at coldfusion.thread.Task.run(Task.java:131) at coldfusion.scheduling.ThreadPool.run(ThreadPool.java:201) at coldfusion.scheduling.WorkerThread.run(WorkerThread.java:71)

                     

                    TagContext        

                     

                    array

                     

                    Type      java.util.ConcurrentModificationException

                     

                    NAME   T7

                     

                    OUTPUT              

                     

                    PRIORITY             NORMAL

                     

                    STARTTIME         {ts '2015-12-20 19:06:13'}

                     

                    STATUS                TERMINATED

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                    *************

                     

                    ,'mm/dd/yyyy') start_the_clock_date,

                     

                            sum(piece_cnt) AS BLUE_tot, sum(INCL_SVC_VAR_MINUS4_CNTINCL_SVC_VAR_MINUS3_CNTINCL_SVC_VAR_MINUS2_CNT) as blue,

                     

                            0 AS ORANGE_tot, 0 as orange,hyb_svc_std_desc 

                     

                            from V_mstr a

                     

                            inner join BI_MSTR_SVC_STD b

                     

                            on b.ORGN_ENTERED = 'N'

                     

                            and b.ML_CL_CODE= a.ML_CL_CODE

                     

                            and b.SVC_STD = a.SVC_STD

                     

                            INNER JOIN

                     

                            V_facility c

                     

                            ON c.fac_SEQ_ID = ORGN_FAC_SEQ_ID

                     

                            inner join

                     

                           (select distinct site_name as area_name,area,district_name,district

                     

                           from v_hier) ad

                     

                            on ad.district = c.district        

                     

                            where a.start_the_clock_date = to_date(