10 Replies Latest reply: Mar 8, 2013 4:05 PM by chris.campbell RSS

    FileStream bytesAvailable 1GB limit?

    HeroJr Community Member

      I'm working with reading very large files, around 5 GB, in an AIR application.

       

      When I open a ~5GB file into a FileStream, bytesAvailable has a value of 1073741824 (0x40000000, exactly 1 GB). I therefore can not read past a 1GB boundary with any combination of .readBytes() and/or setting .position.

       

      Is this an expected behavior? Is there any other way to read the full contents of files > 1GB size?

       

      Below is a sample application. When I select my large file the trace is:

       

      5368709120 1073741824

       

      --

       

      <?xml version="1.0" encoding="utf-8"?>

      <mx:WindowedApplication xmlns:mx="http://www.adobe.com/2006/mxml" layout="absolute" creationComplete="onCreationComplete(event);">

      <mx:Script>

        <![CDATA[

         private var _docsDir:File = File.documentsDirectory;

         private var _stream:FileStream;

         private function onCreationComplete(event:Event):void {

          _docsDir.addEventListener(FileListEvent.SELECT_MULTIPLE, onSelectFile);

          _docsDir.browseForOpenMultiple("Select File");

         }

         private function onSelectFile(event:FileListEvent):void {

          _stream = new FileStream();

          _stream.open(event.files[0], FileMode.READ);

          trace(event.files[0].size, _stream.bytesAvailable);

         }

        ]]>

      </mx:Script>

      </mx:WindowedApplication>

        • 1. Re: FileStream bytesAvailable 1GB limit?
          joafeldmann Community Member

          Same problem here! In my Air (2 beta) app i'm splitting large files to upload them in smaller chunks.

          Everything works fine, until i choose files larger than 1GB? Did you found a solution for this?

           

          One issue is, if i do:

           

          var newFile:File = File.desktopDirectory.resolvePath(filename);                        
          trace(newFile.size);

          // 8632723886  (About 8GB correct file size)

           

          BUT:

           

          var stream:FileStream = new FileStream();

          stream.open(new File(filename), FileMode.READ);

          trace(stream.bytesAvailable)

          // 42789294 ("wrong" file size?)

           

          If i run the same code with files smaller than 1GB stream.bytesAvailable returns the same result as newFile.size.

           

          Is there a limitation in the FileStream class or is my code wrong?

          Thanks!

          • 2. Re: FileStream bytesAvailable 1GB limit?
            chris.campbell Employee Hosts

            Hello,

            Sorry you've run into this problem with FileStream in AIR.   If possible, please give this a try again with the GM version of AIR 2.  You should be able to successfully use FileStream with files up to 4GB in size.  You might also want to give openAsync() a try instead of open(), as that's been reported to work correctly.  Support for file sizes greater than 4GB has been logged as bug #2653204.

             

            Thanks,

            Chris

            • 3. Re: FileStream bytesAvailable 1GB limit?
              GuruMeditation2

              Any update on this one? (I have the same issue)

               

              Or are there any related JIRAs to track this bug?

              • 4. Re: FileStream bytesAvailable 1GB limit?
                chris.campbell Employee Hosts

                Nothing new to report yet, the bug is still open and to be fixed in an upcoming release of AIR.  What size files are you working with?  Have you tried openAsync() to see if that helps?

                 

                Thanks,

                Chris

                • 5. Re: FileStream bytesAvailable 1GB limit?
                  GuruMeditation2 Community Member

                  Thanks for your reply.

                  The file I am testing with is around 6GB but I will need to deal various file sizes for this app going forward. I just made a quick test using openAsync() but couldn't get it working. Will spend some more time at a later point. Will report back if I can't get it in order.

                  • 6. Re: FileStream bytesAvailable 1GB limit?
                    chris.campbell Employee Hosts

                    Here's a post that deals with loading in files using openAsync() in combination with the readAhead property.  It's been used to open very large files and might be applicable here.

                     

                    http://forums.adobe.com/message/3428563#3428563

                     

                    Chris

                    • 7. Re: FileStream bytesAvailable 1GB limit?
                      megharajdeepak Community Member

                      Hi Chris,

                      Wanted to know if there are any similar issues for AIR apps on iOS. (Flex 4.6, AIR 3.6, iOS 6.0.1)

                       

                      I am trying to load a file remotely, it gets loaded successfully. After this, I am trying to read the loaded data into a bytearray using URLStream. Here it fails when we user readBytes. Sometimes it will be successful in reading the files, but again later on when I am trying to write this bytearray data to a local file, it fails again when I use filstream.writeBytes().

                       

                      1. It works fine without any problems when I test it on laptop as AIR app, but fails on the device

                       

                      private var fileData:ByteArray = new ByteArray();

                      public function getFileRemote(pathRemote:String, fileNameR:String):void 

                                  { 

                      urlStream = new URLStream();

                      var urlReq:URLRequest = new URLRequest(pathRemote); 

                      urlStream.addEventListener(Event.COMPLETE, loaded);

                      urlStream.addEventListener(ProgressEvent.PROGRESS, doEvent);

                      urlStream.load (urlReq); 

                      fileName = fileNameR;

                                  }

                       

                      private function loaded(event:Event):void 

                                  { 

                      try{

                      progress = progress + 'before urlStream.readBytes\n';

                      saveLog();

                      urlStream.addEventListener(ProgressEvent.PROGRESS, doEvent);

                      urlStream.readBytes (fileData, 0, urlStream.bytesAvailable); // for file sizes greater than 50MB (upto 800-900MB max in our case), readBytes fails and gets into catch block

                      progress = progress + 'after urlStream.readBytes success\n';

                      saveLog();

                      resolveFile(fileName);

                       

                      }catch(e:Error)

                      {

                      progress = progress + 'catch urlStream.readBytes\n';

                      saveLog();

                      }

                       

                                  }

                       

                      private function resolveFile(pathLocal:String):void 

                                  { 

                      fileLocal = File.applicationStorageDirectory.resolvePath(pathLocal);

                      //if(!fileLocal.exists)

                      //{

                      //fileLocal = File.desktopDirectory.resolvePath(pathLocal); 

                      var fileStream:FileStream = new FileStream(); 

                      fileStream.addEventListener(Event.CLOSE, fileClosed);

                      fileStream.addEventListener(OutputProgressEvent.OUTPUT_PROGRESS, onFileWriteProgress);

                      fileStream.addEventListener(IOErrorEvent.IO_ERROR, onFileWriteIO);

                      try{

                      //fileStream.openAsync(fileLocal, FileMode.WRITE);

                      fileStream.open(fileLocal, FileMode.WRITE);

                      progress = progress + 'zip fileStream.open success\n';

                      saveLog();

                      }catch(e:Error)

                      {

                      progress = progress + 'zip fileStream.open error\n';

                      saveLog();

                      }

                       

                      try{

                      progress = progress + 'before fileStream.writeBytes \n';

                      saveLog();

                      fileStream.writeBytes(fileData, 0, fileData.length);   //writeBytes also fails and gets inside catch block someimtes or most of the times, it crashes on the device

                      //fileStream.writeObject(fileData);

                      progress = progress + 'after fileStream.writeBytes success\n';

                      saveLog();

                      }catch(e:Error)

                      {

                      progress = progress + 'zip fileStream.writeBytes error\n';

                      saveLog();

                      }

                      fileStream.close();

                      sqlite.openDb(fileLocal);

                      //} 

                                  }

                       

                       

                      Can you kindly help? And point out if there any file readBytes\writeBytes limitations on iOS? Or if there are any alternatives for me apart from this code?

                       

                      Appreciate your help.

                       

                      -Deepak

                      • 8. Re: FileStream bytesAvailable 1GB limit?
                        chris.campbell Employee Hosts

                        Hi Deepak,

                        I'm not aware of any existing issues reading files on iOS but I've asked the iOS team to take a look.

                         

                        Thanks,

                        Chris

                        • 9. Re: FileStream bytesAvailable 1GB limit?
                          megharajdeepak Community Member

                          Hi Chris,

                          Well, reading and writing the entire file's bytearray is a problem, mainly because it runs out of memory I reckon.

                           

                          I got an alternative and this works for me

                          http://stackoverflow.com/questions/14583247/air-as3-download-large-file-trough-ipad-applic ation

                           

                          Thanks again.

                           

                          Cheers!

                          Deepak

                          • 10. Re: FileStream bytesAvailable 1GB limit?
                            chris.campbell Employee Hosts

                            Thanks for the follow up.  The iOS is also going to take a look at the source code you provided to see if they can understand why it was failing.