4 Replies Latest reply on May 7, 2010 9:21 AM by lite_As_AIR

    Splitting > 1GB Files // Problem with FileStream class

    joafeldmann Level 1

      Hi, in my Air (2 beta) app i'm splitting large files to upload them in  smaller chunks.

      Everything works fine, until i choose files larger  than 1GB?

       

      This might be the Problem:

       

      var  newFile:File =  File.desktopDirectory.resolvePath(filename);                        
      trace(newFile.size);

      //  8632723886  (About 8GB correct file size)

       

      BUT if i use the FileStream Class instead:

       

      var stream:FileStream = new  FileStream();

      stream.open(new File(filename), FileMode.READ);

      trace(stream.bytesAvailable)

      //  42789294 ("wrong" file size?)

       

      If i run the same code with files smaller  than 1GB stream.bytesAvailable returns the same result as newFile.size.

       

      Is there a  limitation in the FileStream class or is my code wrong?

      Thanks!

        • 1. Re: Splitting > 1GB Files // Problem with FileStream class
          lite_As_AIR

          use asynchronous file handling method. i.e use (filestream object).openAsync(file,filemode.read). here is the implementation :

          private var fileCounter:int = 0;
          private var bytesLoaded:int = 0;
          
          
          private var filePath:String = "D:\\folder\\";
          private var fileName:String = "huge_file";               
          private var fileExtension:String = ".mkv";
          private var file:File = new File(filePath+fileName+fileExtension);
          
          //split size = 1 GB 
          private var splitSize:int = 1024*1024*1024;
          
          private var fs:FileStream = new FileStream();
          private var newfs:FileStream = new FileStream();
          private var byteArray:ByteArray = new ByteArray();
          
          private function init():void{
               fs.addEventListener(Event.COMPLETE,onFsComplete);
               fs.addEventListener(ProgressEvent.PROGRESS,onFsProgress);
               
               newfs.open(new File(filePath+fileName+fileCounter+fileExtension),FileMode.WRITE);
               fs.openAsync(new File(filePath+fileName+fileExtension),FileMode.READ);
          }
          
          private function onFsComplete(e:Event=null):void{                    
               fs.readBytes(byteArray,0,fs.bytesAvailable);                    
               newfs.writeBytes(byteArray,0,Math.min(splitSize-bytesLoaded,fs.bytesAvailable));
               
               for(var i:int = 0; i < byteArray.length; i+=splitSize){
                    newfs.close();
                    newfs.open(new File(filePath+fileName+fileCounter+fileExtension),FileMode.WRITE);
                    newfs.writeBytes(byteArray,i,Math.min(splitSize,byteArray.length-i));
                                                  
                    fileCounter++;
          
                    trace("Part " + fileCounter + " Complete");
               }
          }
          
          private function onFsProgress(e:ProgressEvent):void{
               if((bytesLoaded+fs.bytesAvailable)==file.size){
                    onFsComplete();                         
               }
               else if((bytesLoaded + fs.bytesAvailable)>=splitSize){
                    fs.readBytes(byteArray,0,splitSize-bytesLoaded);
                    
                    newfs.writeBytes(byteArray,0,byteArray.length);
                    newfs.close();
                    
                    bytesLoaded = fs.bytesAvailable;
                    fs.readBytes(byteArray,0,bytesLoaded);
                                             
                    fileCounter++;
                    newfs.open(new File(filePath+fileName+fileCounter+fileExtension),FileMode.WRITE);                         
                    newfs.writeBytes(byteArray,0,byteArray.length);
                    
                    byteArray.clear();
                    
                    trace("Part " + fileCounter + " Complete");
               }
               else{
                    bytesLoaded+=fs.bytesAvailable;
                    fs.readBytes(byteArray,0,fs.bytesAvailable);
                    newfs.writeBytes(byteArray,0,byteArray.length);
                    
                    byteArray.clear();     
               }                    
          }
          

          cheers!

          • 2. Re: Splitting > 1GB Files // Problem with FileStream class
            Jeff Battershall Level 1

            Are you guys talking about the same thing?  The first post seems to be about

            uploading file chunks and other is about downloading.  Is there a way to

            upload from AIR using the FileStream class??

             

            Jeff

            • 3. Re: Splitting > 1GB Files // Problem with FileStream class
              joafeldmann Level 1

              @Jeff: You can write each FileStream-Chunk in a temp file and then use the file - upload() method.

              @lite_as_Air: I'll give the async method a try.

              But can someone confirm the problem i mentioned in my first post? Is there a file size limit?

              • 4. Re: Splitting > 1GB Files // Problem with FileStream class
                lite_As_AIR Level 1

                and other is about downloading.

                 

                the post is not about downloading ... it is about splitting a file into parts of (splitSize <- a variable in the script )bytes length using asynchronous file handling method... (tested on a 2.65 GB file and split it into 3 parts, 2 parts of size 1 GB and the last one of size 0.65 GB duh!)

                 

                the difference between synchronous and asynchronous file handling is that in the former the CPU (air application) waits for the file operation (IO operation) to be performed before continuing the execution of the rest of the script... i.e the cpu keeps waiting for the harddisk to fill the memory after which it continues the execution of the script this would be disastrous if you are going to read a 8gb file! (out of memory error)

                 

                In the later (asynchronous file handling) method the cpu does not wait for the file operation to complete, rather an event listener is installed which alerts the application of the completion of the event (file operation). So the cpu keeps executing the script while data is being transfered from the harddisk to the memory (this data can be retrieved in packets of size approx. 40960 bytes using a progressEvent handler)..this approach is faster and does not run out of memory irrespective of file size if used correctly*. the only downside to asynchronous file handlings is more code..

                 

                therefore it is not recommended at all to use the synchronous approach with file sizes greater than 10 MegaBytes!! the system gets hung up if you try performing (filestream object).readBytes() operation on files > 200 MB and it gives out of memory error if the size is more than 500MB that.

                 

                upload from AIR using the FileStream class

                why bother? refer to the FileReference class here ==> http://www.developphp.com/Flash_tutorials/show_tutorial.php?tid=6 is a detailed tutorial on

                uploading a file with help of php support.

                (how ever with the introduction of the sockets class in air 2.0  it is possible to upload a file to a web server without using php support, i guess it is possible to create a ftp client with air and that way you would not need php ).

                 

                 

                But can someone confirm the problem i mentioned in my first post? Is there a file size limit?

                The (filestream object).bytesAvailable returns the file size for me in case of synchronous file handling even if the file size is 3.04GB but it gives an out of memory error if i perform a (filestream object).read(,,) operation! so why bother about using synchronous approach anyways?