xmlparse works in 8 not in 9

Report · May 06, 2010

I have a client with a store front. Products come from an xml feed. It works perfectly in CF8. Now we are trying to upgrade to CF9 and the parsing doesn't work.

Here is the code

<cfhttp url="http://sunrisewholesalemerchandise.com/swInventory.xml" method="get" />
<cfif cfhttp.responseheader.status_code EQ 200>
<cfset begin = #Find("<?xml",cfhttp.FileContent)#>
<cfif begin>
<cfscript>
   begin = begin -1;
   xmlDoc = RemoveChars(CFHTTP.FileContent,1,begin);
   x = XMLParse(xmlDoc);
</cfscript>
</cfif>
</cfif>

If I cfdump cfhttp.filecontent I see the xml file. But when it hits the xmlParse(xmlDoc) it gives me a blank. This exact code works on the current site that is CF8.

Report · May 06, 2010

x = XMLParse(xmlDoc);

But when it hits the xmlParse(xmlDoc) it gives me a blank.

What do you mean by this? The line you reference doesn't output anything. Do you mean if you subsequently do a <cfdump> of x then it says [empty string], or something?

BTW, why do you do all that horsing around taking the XML tag out of your content? The string should parse as XML fine with it in there.

--

Adam

Report · May 06, 2010

sigh.

Can you slap yourself in the forehead and say "DUH!"

Of course that line doesn't. What did I say about cfdumping before that line? So I stupidly assumed you would know that if I cfdump the output of that line ie x it gives me nothing. The xml is getting lost in the parsing.

Now anybody who has done work with XML feeds for any time at all knows that sometimes the feed doesn't come across the way you expect it to. Sometimes there are vile little blank spaces before the first "<" of the file. That strips them out. Saves a lot of headaches. The question would be more "Why aren't YOU doing that horsing around?" First time you spend hours trying to find out why you're getting errors from the xml parsing you will wish you had done this to begin with.

BUT, I can see this coming so will head it off, I did try it without that "horsing around" and it still didn't work.

So back to the actual question. Is there something different in xmlParse in cf9 from cf8 or maybe a setting in the server that the hosting company set?

Report · May 07, 2010

Can you slap yourself in the forehead and say "DUH!"

I dunno. You're the one posting code with no outputs in it saying "nothing outputs". My response to that was pretty much "DUH", yeah ;-). I have long since learned to assume nothing about the capabilities of the people posting here, or the accuracy of what they're claiming, unless I can see it in front of me.

And to be frank, given you don't even seem to know when to use pound signs, I pretty much assume you don't know much else about CF either. Sorry to be blunt.

What does this give you? I presume you've tried this, but humour me.

xmlDoc = RemoveChars(CFHTTP.FileContent,1,begin);

writeDump(var=xmlDoc, label="After removeChars()");

x = XMLParse(xmlDoc);

writeDump(var=x, label="After xmlParse()");

Can you also pls post some XML which we can test on our own systems to see what we get?

You ask if there's a setting a hosting company can switch off... this suggests that this only happens on your production server? Does it not happen in your dev environment?

Also, you say the same code works on CF8... are you testing it with the same feed data?

What does isXml() give you, in both situations (IE: CF8 & CF9)?

I don't know of anything that's changed in the XML parsing between CF8 and CF9... but if you can provide a portable reproduction case, it'll make investigation easier/possible. Plus if there's a bug, it'll be something to take to Adobe and go "look... bug!"

Now anybody who has done work with XML feeds for any time at all knows that sometimes the feed doesn't come across the way you expect it to. Sometimes there are vile little blank spaces before the first "<" of the file. That strips them out. Saves a lot of headaches. The question would be more "Why aren't YOU doing that horsing around?" First time you spend hours trying to find out why you're getting errors from the xml parsing you will wish you had done this to begin with.

Yep, good point. Sorry, I misread your code last night, and thought you were lopping off the actual <?xml?> tag. Tell me though... what happens if the XML doc doesn't have one of those? It is optional (recommended, sure, but not required). Where did you have your dumps when you were finding x to be empty? Was it within the if() block, immediately after xmlParse() call? Or at the bottom of the code? Are you sure the if(begin) is true? Also, on that subject, I'd probably only bother doing the trimming if begin was > 1. If it's 1 it implies the <?xml?> tag is present, but is at the beginning of the string, so there's actually nothing to trim. Not that this has anything to do with your issue, but it's slightly more efficient logic.

--

Adam

Report · May 07, 2010

Well, I agree with Adam that a little more detail on exactly what procedures you've tried would help us understand how to look at your problem. My first question would be to ask what happens if you try to execute an xmlParse(cfhttp.filecontent) immediately after the CFHTTP call, and then CFDUMP that out. Does the xmlParse give an error? Is it possible that the clean-up code you have in there to try and remove extraneous text at the front of the doc is wiping out too much of the doc? Maybe output some info on how many passes it makes through the loop. I've seen cases where there are exactly 2 extraneous characters at the very beginning of the content, before the opening < of the xmldoc, but never more than that.

What happens if you CFDUMP the unparsed contents of the var that xmlParse() is saying contains an empty doc? That should give you a clue as to whether the problem is with your clean-up code or the parser. Be sure to use CFDUMP and not CFOUTPUT to do this, or you won't see much of anything on your screen!

The answer to your question is "yes" - the xmlparser is a newer version with CF9. Check out these links for prior discussions on this topic:

http://www.mail-archive.com/cf-talk@houseoffusion.com/msg344988.html

We're all anxious to hear more on just where your problem is, good luck!

Reed

Report · May 07, 2010

Will somebody PLEASE read what I have written?

Let me recap what I've already put in here.

1 - Doing a cfdump of filecontent gives me the xml file in one big long string as expected

2 - dumping AFTER the xmlparse with and without the extra code gives me NOTHING

The file is big so I can't post it here but you can go to the exact source because I posted it. Just take it out of the cfhttp if you want to see it.

Adam, which #'s are you talking about? Since where to use them is always open to debate and I've tested both with and without and leave them where they are necessary (even if you don't think so) AND the extra code was actually taken from Ben Nadel .... what are you talking about? Yes, I can tell you are still kind of a noob at ColdFusion if you don't realize it doesn't always do what the documentation says it is supposed to. Like the #s. Sitting screaming at the screen "I DON'T NEED THEM THERE" while your code is not working does not work. Just put them in and let CF have its way.

Reed, if the clean up code was removing too much I would get an error and it wouldn't work in CF8 either. No errors just a blank screen when I cfdump. Not cfoutput. I said that didn't I? The part of the cfdump?

Ya know. People get on me for being a bit snarky at times but this thread is a perfect example of why. People not reading what is written. People jumping in with snarky answers right off the bat. Adam could have replied "I'm assuming there is code after this that is supposed to output the xml" instead of getting rude.

I always have said "RTFQ" first. (Read The Fraking Question) So far, it appears, nobody did. But thank you Reed for the link. I'll check it out.

Report · May 07, 2010

I believe you said that you did a CFDUMP of the cfhttp.filecontent. I believe that I asked what the results were if you did a CFDUMP of xmlParse(cfhttp.filecontent) immediately following the CFHTTP call.

-reed

Report · May 09, 2010

Who's not reading whom here? I asked a bunch of relevant questions (as did Reed), and you've completely ingoring them, instead focusing on a comment I made about pound signs which isn't relevant to the situation. Your ignorance of when (not) to use pound-signs was just an indication that you're not a terribly experienced CF developer, so I figured it best to start with at as basic a level as possible, and not make any assumptions. That said, I did ask you for a portable repro case which you had indeed already posted, so that was a bit thick.

Anyway, Reed's right: the XML is parsing fine (I have to concede I did not bother actually running your code to see what happened). You can demonstrate this to yourself by simply outputting it (<cfoutput>#x#</cfoutput>).

I guess I assumed that when you said it gives you a blank that <cfdump> was saying "empty string", not that <cfdump> itself was failing. Given that <cfdump> was outputting nothing it should have been obvious that <cfdump> was choking? But I should have run your code rather than assuming.

You should probably raise the <cfdump> bug with Adobe though. http://cfbugs.adobe.com/cfbugreport/flexbugui/cfbugtracker/main.html

--

Adam

Report · May 09, 2010

Well, Adam, you obviously are clueless since you think snide comments should be ignored. As I said, YOU show a lack of expertise in CF a lot in your answers. You assume too much and think it (CF) always works by the book. Since I've been using CF since version 4 building big and little applications (right now one that will be used nationwide by the largest employer in the country) I'd say I have just a tad bit of experience. But since I'm just diving into CF9 now and couldn't find anything about xmlparse changing, I thought I would ask. I'm sure tho you NEVER have to ask those questions.

Now then, I have this exact script up and running on a GoDaddy host. It runs fine on CF8 (GoDaddy). Before that it ran fine on CF7 (I think that was CfDynamics). I moved it to 8 with absolutely no problems at all. So it was a surprise to have it die on 9. Giving NO OUTPUT ie nothing, nada, no errors, no comments nothing. The host for this version is giving 1G RAM to the site. I'd be amazed if that is less than GoDaddy but maybe it is. However I have seen xmlparse fail on other sites due to the size of the file and it gives an error. If I remember right some 500 error. But possibly the new host has a limiter on memory that shuts down the running process if it hits a certain limit and this is hitting it.

What this does is populate a database table. Right now it is not doing that.

If choking is the problem, then I will have to first figure out how to split the file before parsing it. Could be a pain.

Report · May 07, 2010

This *is* an interesting problem in its own perverted way. I have reproduced your exact behavior on CF9. For what it's worth as another datapoint, when i tried to run it on a CF8 server (which routinely parses about 100,000 xml docs a day with CF, many of them much much larger than yours), it gets a JVM out of memory error.

But I don't think you have a problem - this appears to be a bug in CFDUMP handling the large xml document. I think that your XML doc is intact. When I added this code at the end of your trimming code, it correctly listed the first 5 productIds in your file:

<br>

Array length=#arraylen(parsedDoc.inventory.product)#

<BR>#parseddoc.inventory.product.itemID#

</cfloop>

Output:

Array length=5336
4311-CD
20302
20679
21163
21310

Debugging Information

ColdFusion Server Developer	9,0,0,233019
Template	/reed/testforum1.cfm
Time Stamp	07-May-10 04:03 PM
Locale	English (US)
User Agent	Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1045 Safari/532.5
Remote IP	127.0.0.1
Host Name	127.0.0.1

Report · May 09, 2010

For what it's worth as another datapoint, when i tried to run it on a CF8 server (which routinely parses about 100,000 xml docs a day with CF, many of them much much larger than yours), it gets a JVM out of memory error.

Yeah, I got the same. My standard config has 512MB allocated to the heap, and running with that CF face-plants after a few minutes. I upped it to 1024MB and it runs for a few more minutes and face-plants. I gave it 1536MB - about as much as I have spare on this laptop - and it ran at 100% on one of my CPUs for 40min then... face-planted.

So I'm interested in what the OPs config is, and what his definition of "works on CF8" is.

NB: it's the <cfdump> that causes the problem, not the xmlParse().

--

Adam

Report · May 09, 2010

Is there any chance at all that the two of you can stop your incessant bickering, and maybe focus on the problem iteself?

Dinghus, can you confirm (using the code snippet that I posted) whether the CF9 config is able to display any of the data from the result of the xmlParse() call? As i pointed out, don't use the CFDUMP (due to the size of the document) - just run through that short CFLOOP. This should help show whether the parsing generated a good xml variable, without introducing other factors.

It might be useful to put the xmlParse() call inside of a CFTRY/CFCATCH, just for grins and giggles, to see if there is some error being generated that for some reason the CF9 configuration is not otherwise reporting.

Here's another shot in the dark: What if the CF9 config has a site-wide error handler defined in CF Admin, but the script that it points to isn't displaying the error details? Maybe try running a small CF script that has an obvious error, just to see if anything is displayed. It's a long shot, but this is a pretty odd-ball problem to begin with. Maybe create a bad XML file and try to run it through xmlParse() - but you'll probably have to do that by hand, not with the CFXML tag, otherwise it will be the CFXML tag that errors-out and you won't get to the xmlParse() call.

But please, really, let's just stick to solving this problem, OK?

thanks,

-reed

Report · May 10, 2010

Is there any chance at all that the two of you can stop your incessant bickering, and maybe focus on the problem iteself?

What bickering? I can't help it if my rationale for assuming - perhaps incorrectly - Dingus was a noob and aimed my initial line of confirming his issue slightly too low for his liking, and it rankled him. I think my rationale was sound enough.

Anyway, we've both made some suggestions (all ignored thusfar), and - to me - I thought we'd identified that the problem isn't with xmlParse(), it's with <cfdump>. So really... not a problem.

I see this whole blank screen thing too: but if one does a view-source, one can see that it's the <cfdump> that's not working. It's not the xmlParse() that's failing. If one instead simply uses <cfoutput> to output the XML, there it is (or your tactic of verifying it is an even better one, because one has to be able to do more with the XML than just view it!). All I can think is happening here is that the code of <cfdump> has had some out-of-memory handling code added to it which suppresses the error that both you & I were seeing on CF8. Hats-off to Dingus that he can get the dump to work on CF8: I guess he's running on a proper server with a decent spec.

However Dingus seems rather more intent on histrionics than actually reading what people have said (irony there), so... dunno what further to suggest. Well... I've a few ideas, but none of them are constructive.

Although I'll repeat your question to Dingus: have you actually tried either of Reed or my own suggestions that should (hopefully) demonstrate to you that you perhaps don't have a problem. With the XML, I mean.

--

Adam

Report · May 10, 2010

Here is what I did to find the problem finally. I downloaded the xml file onto a local drive and tried it from there. Still nothing.

The code after the xmlparse is a cfloop over the xml putting the data into a database and nothing was going into the db. I did try just doing a cfloop over like the first few and this came back blank.

Then I just did a cfoutput of the whole xmlparse'd data and I got something. But it locked up the page. I then started taking it chunks at a time and that worked until I hit 1 bit of data in there. For some reason on 1 of the products an ASCII code is causing the whole thing to lock up. I'm not even sure what the ascii does. HEX 000011 (DC1). I'll look that up later out of idle curiosity. So I do a search for that on the raw data and delete it and now everything works fine it would appear.

Report · May 10, 2010

That's weird. Having one of those characters in the file should invalidate it right off the bat: that renders the file "not XML".

If I run this code:

And then browse to data.xml (like in Firefox) I get an error, as one would expect. However if I browse to that feed you supplied earlier, Firefox parses it fine.

ColdFusion Builder refused to open the file (I let it try for 5min, but then had to restart it), and my other text editor could not find any 0x11 character in the file either.

When you were looping over your XML (trying to put it in the DB), and you say nothing was going into the DB was this because your loop simply wasn't iterating, or was it something else? IE: if you output some plain text in your loop, did it output, or was the looping condition never being met?

--

Adam

Report · May 14, 2010

Okay, the final answer is ......

It was a host problem.

I am posting this in case anybody else runs into this one. The previously thought solution didn't sit right so I kept digging. Then I had another problem with CFINCLUDE not working. This was traced to a problem with the sandbox. AH HA!

What the problem turned out to be is the sandbox wants only lower case domain names. The domain was www.MyDomainName.com (example only). So when it tried to xmlparse or cfinclude it was failing to do anything because it was looking for all lower case letters and was case sensitive.

And it is a Windows host.

We redid the host name on the hosting server and everything is now working just fine.

Report · May 14, 2010

Okay, the final answer is ......
It was a host problem.

Wow, that's weird!

I can't even see the corelation between domain names and <cfinclude> and xmlParse(). Odd.

Good that you got to the bottom of it, though.

--

Adam

Adobe Community

xmlparse works in 8 not in 9