Copy link to clipboard
Copied
I need to routinely copy a .png file from a remote server that is behind a firewall.
Copy it to the local coldfusion server.
I know cffile is restricted to copying objects on the local server, so how do I go about "scraping" like this.
Do I use cfhttp?
Many thanks
R
Copy link to clipboard
Copied
Yes.
Copy link to clipboard
Copied
Ok, so yes ... could you shed light as to HOW?
R
Copy link to clipboard
Copied
I'd assume you use the CFHTTP tag and get the image, what have you tried so far and what have you found?
Copy link to clipboard
Copied
So you use the cfhttp tag .. using what fornat to grab the image?
The once you have it, how do you create the local file cffile?
Or is it built into cfhttp?
Not exactly documented anywhere.
R
Copy link to clipboard
Copied
CFHTTP goes out into the world and returns back the data for whatever web resource you are telling it to grab. In the case of web pages, this returns back the raw HTML of the page. However, you are not limited to just asking for HTML pages - you can ask it to retireve ANY web accessible resource and CFHTTP will dutifully go out, grab the data and return it to you. In the case of non-text-based resources, you will be getting back the code that the resource is made up of (binary data, post-script, etc). Once you have that data, you can write it to a file on your system and access it as if it were a local file.
The one thing about your initial statement that raised an eyebrow is that you said the remote server was behind a firewall. I'm assuming that you really mean that the server is physically being protected by a firewall and not that the PNG image is being protected from external reference by some sort of login or access control mechanism. In the case of the former - the existance of a firewall is not an issue as CFHTTP uses port 80 to make regular web requests so in that sense its really no different from your browser asking for the PNG. If there is a log in or access control mechanism in place that is a complete different challenge and may also involve some legal issues related to accessing content that you don't have permission to retrieve electronically.
Hope that helps,
- Michael
Copy link to clipboard
Copied
Thanks ... let me experiment with this further.
Firewall issue ... our web servers in the USA can acess the overseas servers. They have permission. The clients (surfers) accessing the web server cannot. And do not need to know our IP addresses.
So we want to copy the cached images to the local server and refer to them as www.domain.com/images/image.png rather than ???.???.???.???/images/xxx.png
Many thanks .. and will update.
R
Copy link to clipboard
Copied
Well, so far not good.
I used cfhttp and use url="xxx.xx.xxx.xxx/images/xxxx.png" I get an error back file not found when printing out cfhttp.filecontent.
Yet when I paste the URL directly into a browser on the server I get a graph displayed.
I must need a parameter?
R
Copy link to clipboard
Copied
It might be related to the permissions lockdown you were mentioning earlier. Are you performing the browser tests from the same machine that your CF server is running on? They may also be blocking access depending on the user agent making the request. The user agent string (identified in the CGI scope) identifies the type and version of the browser being used. Its possible that they might be blocking access to user agents they don't recognize. I believe there is a way to manipulate the user agent used by CFHTTP, but I've never looked into it so I'm not familiar with the process. If browser requests are working, but CFHTTP is not ON THE SAME MACHINE, then see if you can spoof your current browser agent (dump CGI scope to find this) and see if that works.
Otherwise, there could be some other authentication scheme in place (cookies, headers, etc) that could be interferring. You'll have to use a tool (I think firefox has a plug in) that lets you take a look at the "extra" stuff going on in your request to see if there is another cause.
Also, make sure that you include the protocol (http/https) in the URL for your CFHTTP call.
Copy link to clipboard
Copied
We manage both servers so I'll check with my IT manager and see how the firewall is configured.
Yes, browser and cfhttp on same machine.
R
Copy link to clipboard
Copied
This thread on Stack Overflow may get you where you need to go:
http://stackoverflow.com/questions/2942290/how-to-emulate-a-real-http-request-via-cfhttp
Copy link to clipboard
Copied
Wow.
Almost got it working.
I cheated and copied all those parms ... executed it and in the destination I put "c:\inetpub\wwwroot\appname\coldfusion\graphs and named the file example_graph.png
No errors, but no file either ... searched server and found the file hiding in
c:xxxx\coldfusionmx7\runtime\servers\coldfusion\server-inf\temp\wwwroot-tmp.
So ... 99% there ... what did I miss?
R
Copy link to clipboard
Copied
To save the response to a specific location, you need to use the "path" and "file" attributes. Path is the directory (only) where CF will save the response, and file is the destination file name.
http://help.adobe.com/en_US/ColdFusion/9.0/CFMLRef/WSc3ff6d0ea77859461172e0811cbec22c24-7ffc.html
Copy link to clipboard
Copied
I have it working .. almost ... the .png now saves to the proper location ... (I knew that! ) I added the proper mapping in cffile
And I can see the "reference" to it in a URL accessible ot the public.
But when I ask the browser to display it, it gives a broken icon.
And when sing Windows Explorer and open it on the server, it is broken. So it must not be saving properly?
R
Copy link to clipboard
Copied
Not sure what you mean by broken or where CFFILE comes into this. Can you post some code?
Copy link to clipboard
Copied
By broken I mean ... the traditional "Explorer" broken icon. I.e. it is finding the object ...
Ok, WEIRD.
In the URL in m browser I provide the correct address and I get back .
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /graphs/graphs/graph_139_1.png was not found on this server.</p>
<hr>
<address>Apache/2.2.3 (CentOS) Server at 80.4.157.12 Port 80</address>
</body></html>
This is when I use IE. But in the URL I am pointing to an address ... http://www.domain.com/graphs/real_example.png NOT 139_1, which is the original name of the file I copied !
When I use Firefox I get an image back ... which in text says ...
The image [correct image URL] cannot be displayed because it contains errors.
The code I use is.
<cfhttp | ||
url="http://xx.xxx.xxx.xxx/graphs/graphs/graph_139_1.png" | ||
timeout="10" | ||
throwonerror="No" | ||
useragent="Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/533.7 (KHTML, like Gecko) Chrome/5.0.391.0 Safari/533.7" | ||
> | ||
<cfhttpparam type="header" name="HTTP_REFERER" value="http://example.com/feed/" > |
<cfhttpparam type="header" name="Accept-Encoding" value="gzip,deflate,sdch" >
<cfhttpparam type="header" name="Proxy-Connection" value="keep-alive" >
<cfhttpparam type="header" name="Accept" value="application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5">
<cfhttpparam type="header" name="Accept-Language" value="en-US,en;q=0.8">
<cfhttpparam type="header" name="Accept-Charset" value="ISO-8859-1,utf-8;q=0.7,*;q=0.3">
<cfhttpparam type="cookie" name="some-cookie" value="1">
</cfhttp> |
<cffile action="WRITE" | |||
output="#cfhttp.filecontent#" | |||
file="c:\inetpub\wwwroot\xxxxxx\coldfusion\graphs\real_example.png" | |||
> | |||
<img src="http://www.xxxxxxxxxxx.com/graphs/real_example.png"> |
Copy link to clipboard
Copied
Have you made sure that the account that coldfusion is running under on your server has access to write to that directory? If you pull up the file details in windows explorer for that file, do you see anything out of the ordinary (size:0, etc)?
Copy link to clipboard
Copied
Nothing unusual ... file size shows 4k.
What I'd expect.
R
Copy link to clipboard
Copied
I can only assume that CF is writing the file to the disk wrong ... but with nothing else to go by, we declare this a fail grade
R
Copy link to clipboard
Copied
cfhttp getAsBinary ?
--
Mack
Copy link to clipboard
Copied
Good idea maybe but now I get new error messages.
The web site you are accessing has experienced an unexpected error.
Please contact the website administrator.
The following information is meant for the website developer for debugging purposes. | |||
Error Occurred While Processing Request | |||
|
Yet the path is in perfect condition.
And the code is:
<cffile action="WRITE" | |||
output="#cfhttp.filecontent#" | |||
file="c:\inetpub\wwwroot\xxxxxx\coldfusion\graphs\real_example.png" | |||
> |
xxxx may not be correct length ... hiding client name.
Copy link to clipboard
Copied
This looks like a permissions issue.
Insuractive wrote:
Have you made sure that the account that coldfusion is running under on your server has access to write to that directory?
Copy link to clipboard
Copied
Checked perms on the folder ... allows all rights to admin, internet guest account (cfusion), iis process account,
But weird bit is when I do this
<cffile action="WRITE" | |||
output="Test" | |||
file="c:\inetpub\wwwroot\xxxxxx\coldfusion\graphs\real_example2.txt" | |||
> |
An error occurred when performing a file operation WRITE on file c:\inetpub\wwwroot\xxxx\coldfusion\graphs\real_example2.txt. | |
The cause of this exception was: java.io.FileNotFoundException: c:\inetpub\wwwroot\xxxxx\coldfusion\graphs\real_example2.txt (The system cannot find the path specified). |
Which could not be more simple.
Copy link to clipboard
Copied
When performing a <cffile> WRITE, if a file does not exist CF automatically creates a file. The only time you should see an error like the one above is if for some reason the CF/Java service is unable to access the file directory structure that you are trying to write to. The most likely issue is something to do with permissions, but its possible that there is a problem with something in your CF installation that is preventing the write. Try restarting your CF service and see if you can perform a test <cffile> write.
Can you perform a write in another directory?
My suspicion, though, is that we're still looking at some sort of insidious permissions error.
Copy link to clipboard
Copied
allows all rights to admin, internet guest account (cfusion), iis process account
But none of those are the account ColdFusion is running as I'd guess - IIS account is *not* necessarily the CF account. Check the services console to see what user it's running as.