3 Replies Latest reply on Nov 10, 2009 4:09 AM by Santanu Karar

    File Chunking on upload

    flashharry! Level 1

      I have to solve a problem of uploading large 4gb files.

       

      Anyone have any experience or point to an article on the following,

      With a file reference we can get to the fileReferance.data and load it into memory.

      I wonder if you could part load a chunk of data say 80mb, upload that 80mb, then discard that 80mb and load the next 80mb

      You would loop until end of data, I can reassemble the data at the server not an issue.

      The trick here is that I do not want to load the entire file into memory of the flash player just a chunk at a time and discard the uploaded chunk to free up the memory.

      The idea would be to build on this is to be chunking the next bit of data while the previous chunk is uploading, also resumable uploading.

      If we start the upload again, start so many chunks (bytes) into the file.

      When we start the file process take the  file size from the file reference and divide by the chunk size, when we send the first chunk we tell the server how many chunks of data to expect.

      Thoughts, ideas appreciated.

      TIA

        • 1. Re: File Chunking on upload
          hoolmx

           

          not sure if you can do this outside of AIR, but i've done it as an AIR app.

           

          use FileStream to openAsync() a file for FileMode.READ

          use FileStream's .readAhead property to define the amount of bytes you want to read in at a time

          define your own chunkSize value for the size of the chunked files

          use the ProgressEvent fired by FileStream to let you know when enough bytes are available to fill a buffer of chunkSize

          create a new File and write those chunkSize bytes into it

          when that file has finished writing to the local disk, use File.upload() to upload it to your server

           

          repeat until all chunks are there.  using openAsync with a smaller chunk size means you don't have to load the entire file into memory.  using File.upload() gives you upload progress.  (Sockets are still broken in as3 in that they don't report the progress of the file)

           

          it's quite simple to set up your own mini protocol for resuming broken transfers.  just keep track of the last chunk index that was successfully uploaded, and start from that index to resume a broken upload.  reassembling the chunked files back into the original file on the server side is trivial.

           

          i don't think this will work without a lot of user intervention in a flash app running inside a browser, due to flash's security restrictions.  with flash 10 you can write to the local disk, yes, but i believe manual user intervention/permission is required each time.  furthermore, i don't think you could upload() the chunk files that were dynamically created without manual user intervention/permission for security reasons.

           

          1 person found this helpful
          • 2. Re: File Chunking on upload
            flashharry! Level 1

            I was trying to keep this in the browser, I may be force to go to java on this, although AIR would work most of our corporate clients are not allowed to install desktop apps with out involving their IT department

            I was hoping for some extra call in load like read ahead and part load the file in memory and  upload that part read.

            • 3. Re: File Chunking on upload
              Santanu Karar Level 1

              Hi Hool,

               

              Will appreciate if you could share the piece of code chunking the files in AIR with us, as the Adobe's example code for readAhead seems worthless. Really appreciate if you could share.

               

               

              Thanks.

               

              Santanu.K