We cache the results to some long running queries. Makes sense, except if the caced query is called a second or third time before the first one has been returned.
So lets say the first query takes 1 minute to complete. During the next 59s a user may try and repeat the same query over and over again. This casues CF to spin full speed and take 1 core per request to 100%. It appears to be a loop with no latency or sleep just anxiously waiting for the first cached to be completed. Once the first query is completed, all of the other threads come to completion and the cpu drops back down to our baseline activity level (~1-3%)
Sadly this event will max out the server and we see CF stop responding to new requests at the same time this is occurring, and basically this can lock up the server until the original query completes. The server is fine after the event is over, but this issue has been seen in cf7 and cf8 servers. Has anyone else ever seen this or does anyone know how to slow down the second thread while waiting for the query to complete.
Maybe we need to write our own caching object to replace the CFquery's object since we know it cannot handle this scenario.
You might want to try putting a named read only cflock around the query to prevent simultaneous calls.