3 Replies Latest reply on May 3, 2006 10:10 AM by Mr Black

    Need Help w/ CFHTTP to set Cookies

      I am trying to use CFHTTP to get data from an online journal source.
      The problem is that it need to set alot of cookies and do alot of redirects.
      I want to make a "Web Service" from this and just pass ISSN, Volume, Issue, Page
      to the website and scrape the data that comes back, but I am having problems with "session time outs" etc returned.
      I did a webtrace using CHARLES (great software btw...) and this is what happens in the browser (see below)

      I need to get ColdFusion to do the following:

      Thanks if you can help with this.

        • 1. Re: Need Help w/ CFHTTP to set Cookies
          Mr Black Level 1
          If you under Windows, get this CFX_HTTP5 and do it in one shot.
          • 2. Re: Need Help w/ CFHTTP to set Cookies
            DaveF67 Level 1
            Thanks, but no. I'm not trying to get 4 different URL's,
            I am trying to get 1 URL that gets bounced 4 times.

            I ahve tried this: but still no luck:

            <CFHTTP URL=" http://www.scopus.com/scopus/openurl/document.url?issn=10409165&volume=19&issue=1&spage=32 "
            METHOD="GET" RESULT="webGetData" RESOLVEURL="yes" REDIRECT="no"></CFHTTP>
            <CFDUMP VAR="#webGetData#">
            <!--- pick out cookie information and store in temporary structure--->
            <cfset tempCookieStruct ="#webGetData.responseheader['Set-Cookie']#">
            <!--- send the user credentials with previous cookies via HTTP post--->
            <CFHTTP METHOD="POST" RESULT="webGetData2" RESOLVEURL="Yes"
            URL=" http://www.scopus.com/scopus/openurl/document.url?edit.scft=1">
            <CFHTTPPARAM TYPE="cookie" name="Set-Cookie" value="#tempCookieStruct[1]#">
            <CFHTTPPARAM TYPE="cookie" name="Set-Cookie" value="#tempCookieStruct[2]#">
            <CFDUMP VAR="#webGetData2#">
            • 3. Re: Need Help w/ CFHTTP to set Cookies
              Mr Black Level 1
              This is exactly what CFX_HTTP5 provides.

              The URL in your code above bounces to a login page, which means that you have to request it with already some cookies set (after the login).

              Also. If you want make this work, as sequence of CFHTTP requests (possible), you cannot use POST, when trying to emulate a redirtect sequence. Redirect is always GET.

              Also, you cannot set cookies blindly in your "Cookie" header. There are domains and paths rules. Doing this wrong will break the sequence. For example, a cookie that server expects from you are not necessarily the one you've just received, but maybe some other cookie(s) that was already on your computer, when you hit some other page on the appropriate domain/path.

              Good luck.