4 Replies Latest reply on Sep 13, 2007 10:25 PM by BKBK

    Read and Parse Large XML Files

    red04
      I am having trouble w/ my server being able to read and parse a 92mb xml file.

      Currently I am using:

      <cffile action="read" file="file.xml" variable="theXml">
      <cfset myXml = xmlParse(theXml)>

      Is there any suggestions on how to read and parse a 92mb xml file?
        • 1. Re: Read and Parse Large XML Files
          Grizzly9279 Level 1
          Wow, I've never tried to parse an XML file quite that large before. Does it contain a bunch of encoded binary data, or does it have houndreds of thousands of xml-nodes?

          In any event, I do have one thought:
          Do you need absolutely everything in the file? If you only need access to a small portion of it, it would be worth your while to pull out only the "stuff" that you need before you parse it. You could use regular expressions to strip out things you don't need, or to pull out only the stuff you do need.

          The more you can minimize the "parse" effort the better off you'll be.

          Alternatively, if it's a file describing many "records" of the same type, it would be best if you could "chunk" the file and parse each record individually.
          • 2. Re: Read and Parse Large XML Files
            Level 7
            There are relatively simple ways to dive into the Java underpinnings of
            modern CF servers and read large files such as this line by line,
            instead of trying read and digest the entire thing in one bite.

            But the XML aspect of this data may complicate things since the nature
            of xml data may make it challenging to parse the file bit by bit. I
            suspect the built in CF XML functions are going to want a complete data
            set with all appropriate opening and closing tags to work. You may need
            to roll your own or something, I'm not sure.

            • 3. Re: Read and Parse Large XML Files
              BKBK Adobe Community Professional & MVP
              > ...to read and parse a 92mb xml file

              The cffile is superfluous here. Just parse the path directly, like this
              <cfset myXml = xmlParse("C:\ColdFusion8\wwwroot\website\file.xml")>



              • 4. Re: Read and Parse Large XML Files
                BKBK Adobe Community Professional & MVP
                Oh, and, please tell us whether you could load the biggie.