cfhttp/cfoutput timeout

[reneemckechnie]

I'm new to this forum - sorry if this isn't the right category.

I'm having trouble figuring out how to run a report that uses cfhttp without timing out the cfoutput tag. I'm about 99% certain the issue is the large volume of data and processing time of the tag I'm using, not an endless loop in the code or anything. I've tested the code with small data sets and it runs fine. I know I can increase the timeout setting, but I think the jobs may take over an hour to run as currently coded, and that just feels wrong. Seems like there must be a better way.

The task that's running is checking for new versions of legislation -- there are 45 possible versions for (by the end of session) up to 3,000 bills. Each version is checked by sending a test from the cfhttp tag, and if it exists, inserts the url into the db. It appears I can get through about 30 bills in 60 seconds.

Code is attached. Am I making some sort of ridiculous logic/novice error? Any suggestions for making this work?

Thanks in advance!

[hlevin]

First make sure that you set RequestTimeout in the url parameter to something very high.

Sounds like your browser may also be timing out which would most likely be the case even if you set RequestTimeout to something huge. I would put <cfflush> in every loop iteration and send progress update to the browser, perhaps it will signal to the browser that data is still coming in and it just might prevent it from timing out.

Henry

[reneemckechnie]

Thanks - that's really helpful information.

What's too high for the timeout setting for the page? I think the entire task would take 30 minutes to run. Is it taxing to a CF server to have a page running for that long?

[mvierow]

Have you tried method="trace" in your http calls? If you aren't interested in the body, don't bother doing a GET.

You could probably consolidate all of those queries down to a single query too, then loop using cfoutput, grouping on "Comp_key". Not even sure "allVersions" is even necessary.

If that still doesn't do it, try using maxrows and startrow in your cfoutput. Maybe add a cflocation to the end with a URL param to pass startrow+maxrows to the next instance. Then just sit back and let it run, you can watch the url update in the browser as the records are processed.