0 Replies Latest reply on Dec 11, 2013 2:00 AM by XIntelligence

    Virtual File System performs badly writing many files.

    XIntelligence Level 1

      Both ColdFusion 9 and 10 showed strange performance curves when it comes to writing a lot of files to the VFS.

       

      I tested the possibility to use the VFS to speed up logging.

      The plan was to bulk-post data to a NoSQL (Couchdb) database,

      since cfhttp has a lot of overhead posting small entries to the log database.

       

      Writing just a few files (10-100) to RAM:// is much faster than writing to the physical disk.

       

      Writing 500 files, the physical and the ram disk perform about equal.

      The example below with 10.000 files, the physical disk is ten times faster!

      Note that there were not out of disk memory problems involved.

       

      Instead I  believe problem is related to the number of files created in a single directory.

      To write a file to a directory with many files already, is much slower.

      If I create 10.000 files in 20 different directories RAM maintains its performance curve in comparison to the physical disk.

       

      Is this a bug or something I missunderstood?

       

      Any thoughts about this?

      Help solving this performance problem is appriciated

       

       

      Example output:

      • RAM Disk Speed Creating 5000 files 17.855s
      • Physical Disk Speed Creating 5000 files 2.591s

       

      Example code:

       

      <cfset n = 5000>
      <cfset pathid= createUUID()>
      
      
      <!---  Creating n files on RAM disk  --->
      <cfset _path = "ram:///#pathid#/">
      <cfdirectory directory="#_path#" action="create">
      <cfset t0=gettickcount()>
              <cfsilent>
                  <cfloop from="1" to="#n#" index="i">
                      <cfset uid= createUUID()>
                      <cfset _filename = "test.#uid#.l">
                      <cffile action="write" file="#_path#/#_filename#" output="">
                  </cfloop>
              </cfsilent>
      <cfset t1=gettickcount()>
      <cfdirectory directory="#_path#" action="delete" recurse="yes">
      <li><cfoutput>RAM Disk Speed Creating #n# files #evaluate( (t1-t0) / 1000 )#s</cfoutput></li>
      
      
      <!---  Running the same test for physical disk  --->
      <cfset _path = "c:\temp\#pathid#">
      <cfdirectory directory="#_path#" action="create">
      <cfset t0=gettickcount()>
              <cfsilent>
                  <cfloop from="1" to="#n#" index="i">
                      <cfset uid= createUUID()>
                      <cfset _filename = "test.#uid#.l">
                      <cffile action="write" file="#_path#/#_filename#" output="">
                  </cfloop>
              </cfsilent>
      <cfset t1=gettickcount()>
      <cfdirectory directory="#_path#" action="delete" recurse="yes">
      <li><cfoutput>Physical Disk Speed Creating #n# files #evaluate( (t1-t0) / 1000 )#s</cfoutput></li>