10 Replies Latest reply on May 28, 2008 9:54 AM by arnie_hm

    Struct of Structs

    arnie_hm
      I build a struct with an entry for each URL on my website. Each entry is a struct that contains some information about that URL. The code that builds this query takes about 5 seconds to run, which is reasonable considering there are 1000's of links.

      During the time the code runs, the CPU hovers at about 75% - 95%. I noticed though, that after the request ends and i get back my message saying its finished, the CPU remains high for the next couple of minutes. After that it settles down again. Why would I see that behavior? Is there some processing ColdFusion is doing behind the scenes with the structures?

      If there is, how do I know when that processing is finished? The structure created is an application scope structure, so there is a lock around the creation of the struct. However, it does not seem like processing is done when the lock releases.

      And it seems like requests that run during that extended processing times are getting ColdFusion errors that look like this "Element FOUNDRESULT is undefined in URLINFO". I know that the structure URLINFO definitely has an element FOUNDRESULT. might these 2 issues be related?
        • 1. Re: Struct of Structs
          Level 7
          arnie_hm wrote:
          > might these 2 issues be related?
          >

          Quite possible, especially with the lock, if it is overly aggressive in
          what is locked. You may have very well singled threaded your
          application at this point creating a bottle neck. In other word no
          other requests can run until this request is done creating this
          structure. While this is going on, other requests are waiting for their
          turn to be processed, but they will only wait for so long, and then
          start throwing errors. And so after this request is done processing CF
          potentially has a long list of other requests, ones that have not timed
          out with an error yet, waiting to process. So it gets right on it,
          apparently taking a couple of minutes to catch up with all the held up work.
          • 2. Re: Struct of Structs
            arnie_hm Level 1
            Thanks Ian,

            a couple of points

            1. It has to be locked while the struct is being loaded, theres no getting around that. but all the read requests have a timeout of 1 second, and they dont throw an error when timing out. I may have to build my own lock to resolve this issue, as there are many lookups into this struct for each page so a timeout of 1 is too long.

            2. It doesnt explain the error i'm getting that an element is missing from the struct when i know its there. i get a dump of the struct when the error occurs, and the element is there.
            • 3. Re: Struct of Structs
              Level 7
              What's the timeout on the lock, and does it throw an exception, or just
              keep going regardless?

              In fact... possibly post the relevant bits of your code so we can see
              what's going on.

              --
              Adam
              • 4. Struct of Structs
                arnie_hm Level 1
                here is some of the code
                • 5. Re: Struct of Structs
                  BKBK Adobe Community Professional & MVP
                  I don't see any reason for the locks. I think it's sufficient to call the functions (without locks) in the correct order.

                  • 6. Re: Struct of Structs
                    arnie_hm Level 1
                    you are probably right, BKBK. the reason i added the locks is because i was getting that error i posted. however, the locking hasn't fixed it.

                    any ideas what would cause that error?
                    • 7. Re: Struct of Structs
                      Level 7
                      > <cffunction name="init">

                      So the instance of the CFC this is in is in the application scope, yes?


                      > <cflock name="seoSettingsLock" type="exclusive" timeout="120">

                      I can see a potential problem here (it depends on the calling code, though,
                      really)... are you controlling how many requests call this code, eg: by
                      checking if all this work needs to be done? I would expect that check to
                      be immediately around the lock. If there's no check, you'll get a bag load
                      of requests queuing up for up to two minutes for this lock to clear. In
                      the mean time if the lock clears, then another request will start
                      reprocessing the code within the lock. Surely you don't want that to
                      happen.


                      > <cfloop query="getURLs">
                      > <cfset urlInfo = structNew()>
                      > <cfset linkInfo.URL = url>
                      > <cfset linkInfo.pageTitle = pageTitle>

                      If this code is in a CFC instance that's in the application scope, you have
                      a potential ptoblem here in that your not VARing any of your variables, so
                      they're global to the [shared] CFC instance, not local to the function. I
                      can't see how this would directly cause what you're seeing, but it is
                      generally an open door for "unexpected behaviour".

                      If you intend these variables to be in the CFC's variables scope, not local
                      to the function, can I recommend you always scope them, to make it clearer.
                      If you had said "variables.urlInfo" I'd surmise you meant it to be in the
                      variables scope, not just putting it there by accident.


                      > <cfset structInsert(this.urlsByStatic, staticURL, linkInfo)>

                      Do you really mean to be putting that in the THIS scope? IE: public to the
                      calling code? I'm not quite sure what you're exposing here, as this
                      variable doesn't seem to match the similar-named this.urls you use
                      elsewhere.


                      > <cfset info = "">

                      I'm running on the notion that the CFC instance is in application scope.
                      So all calls to this method will be sharing this "info" variable.

                      One call to it could be setting it to ""...

                      > <cfif structKeyExists(this.urls, keyURL)>
                      > <cfset info = this.urls["#keyURL#"]>

                      ... whilst another request is here.

                      > <cfset info.foundResult = true>

                      So by the time the line above gets executed, info is "", not the value you
                      were hoping it to be, by setting it on the line above.

                      This is probably the cause of your problem.

                      --
                      Adam
                      • 8. Re: Struct of Structs
                        arnie_hm Level 1
                        quote:

                        So the instance of the CFC this is in is in the application scope, yes?


                        Yes, its stored in the application scope. The init code should not be running very often at all, and i cant have the init code running from 2 requests at the same time, so that lock is needed.

                        quote:

                        If this code is in a CFC instance that's in the application scope, you have ...


                        The variables should be local to the function. Not varing them was a mistake. As you say, its hard to predict the effect it may have.

                        quote:

                        Do you really mean to be putting that in the THIS scope?


                        Yes. the line of code is wrong, it should read

                        <cfset structInsert(this.urls, url, linkInfo)>

                        quote:

                        This is probably the cause of your problem.


                        Reading what you wrote, certainly sounds like a possible source of the problem. I will fix the code and let you know how that works out.

                        Thanks Adam!
                        • 9. Re: Struct of Structs
                          dlevitt
                          Im not 100% sure of what you are trying to achieve but can you not just store the query created in the init function within the application scope and then use the URLinfo function to look at that query (and add/update as needed)?

                          Does this need to be a struct of structs when you can treat the query as one big structure anyway?
                          • 10. Re: Struct of Structs
                            arnie_hm Level 1
                            Yup, that did the trick, Adam. Thanks!