File.copyTo (and copyToAsync) extremely slow on Android. e.g. 1/2 second per tiny XML file. Help?

[AllanPadgett]

On first launch, my app copies about 100 tiny XML files from application dir to storage dir. It takes about 35 seconds. That is clearly an issue. It takes about 1/100th of a second on any other platform. I've switched to copyToAsync() so that the user doesn't bail, but still this is like using a dial-up modem! Anyone have any thoughts?

[Damanjit Singh]

One reason could be that the Android package is a zipped format, thus copying a folder, with multiple files, from app directory, requires extracting the zipped bytes for each file in the folder, unzipping it, and then copying it to the destination.

Though it would be great if you could file a bug with a sample application (with code) do that we could reproduce and analyze at our end.

Thanks,

Daman

[Abhinav_Dhandh]

Hi Allan

I tried to reproduce this issue in which I have copied 100 XML's from application directory to applicationStorage directory in Android AIR app. The issue is not reproducible as the files were copied almost instantaneously within 1-2 seconds.

I have shared my sample project here :  https://dl.dropboxusercontent.com/u/184715790/Shared/FileCopyToTest.zip

Additionally, I have also attached the .apk : https://dl.dropboxusercontent.com/u/184715790/Shared/FileCopyToTest.apk

You could try it out and confirm if the issue is still visible.

Also could you inform about your device, OS version & AIR version which you are using ?

Regards,

Abhinav

[AllanPadgett]

Hi Abhinav -

I notice that my test is Flex. Your test is pure AS3. You are also using the beta 3.8 release, not an official public release?

I will get the 3.8 beta. Could you also build a 3.7.1 Flex test? Or test the one I sent via Gen?

Thanks,

Allan

[jadams602]

Just to chime in that I also am seeing very slow copyTo and copyToAsync performance on Android.

I am using the very latest AIR 3.8 beta SDK July 30th 3.8.0.1040

The use case is something we are commonly required to do on Android:

Showing local HTML help files with a StageWebView.

As you probably know, one cannot load local files on Android from File.applicationDirectory with nativePath because this is null on Android (loading from here without copying is fine on iOS however)

So in order to load local HTML help on Android in StageWebView one needs to first do:

var source:File = File.applicationDirectory.resolvePath("help");

var destination:File = File.applicationStorageDirectory.resolvePath("help");

source.copyToAsync(destination, true);

and then one can successfully load HTML files from the destination dir using nativePath URL

I have about 170 small size files in total under 'help/', a mix of some .html and some .jpeg images used in the .html totalling 6 MB in size

On *both* a Kindle Fire and on a Google Nexus 7, this single call to copyToAsync from one directory 'help' File to a new destination directory 'help' File takes a little over 20 seconds.

Fortunately I only have to do this on first viewing of help in the app, but is the slowness a result of how the compressed directory in the .apk is doing a file by file lookup and uncompress as opposed to being able to extract all files of the parent directory in one go?

Am I better off having the full help actually stored in the .apk as a .zip resource and doing a single file copy of the .zip and then an external uncompress? Or is there a way for your internal access of files within File.applicationDirectory to  optimize a full directory and all its children when the source directory File is in the .apk?

[AllanPadgett]

Hi Jeffery -

The ZIP file / expand externally is what I'm pondering doing too. Unfortunately just haven't had the time. Please let me know if you try it out, however?

Thanks,

Allan

Allan Padgett, CEO

Tour Tracker LLC

email: tourtracker@gmail.com

web: http://thetourtracker.com

twitter: @tourtracker

facebook: http://facebook.com/thetourtracker

[jadams602]

Allan,

Crazy good results to report with the idea of simply adding a .zip of the files as a resource into the .apk, then just doing a copyTo of the single zip file, and externally unzipping the contents within the File.applicationStorageDirectory

I took the exact same help/ folder of 170 files totalling 6 MB+, Zipped it into a single 5MB+ help.zip

Added that help.zip as a single bundled resource for the .apk

Then doing a single file copy

var source:File = File.applicationDirectory.resolvePath("help.zip");

var destination:File = File.applicationStorageDirectory.resolvePath("help.zip");

source.copyTo(destination, true);

followed by a very basic and efficient Java expand zip function using builtin Java File IO classes

and simply placing the extract(sourceZipPath, targetFolder) functionality into an Android ANE for the app.

What originally took 20sec+ on a first gen Kindle Fire and Google Nexus 7

now takes less than 3 seconds on the Kindle File and only less than half a second on a Google Nexus 7!

So huge win in my case.

There must definitely be an expensive issue of doing a mass copyTo with nested files that are from the source zipped .apk  Would be great if Adobe could speed this up, or maybe as a fallback they should add builtin Zip decompression in the AIR API so developers can make easier use of zipped resource file sets.

I see that Adobe staff replied with a sample project where they were not able to duplicate the slowdown.

I was using an 'Export Release Build... ' packaged .apk, so maybe extra compression or slowness results from different packaging modes, who knows, but it is something I am definitely seeing on the latest AIR 3.8 SDK and on multiple Android device types.

[AllanPadgett]

Thanks for the great update. Would you be willing to share / sell me your ANE and sample code? Never built an ANE and with 3 races left this year, don't plan on having the time to learn. Very nice work!

Allan

Allan Padgett, CEO

Tour Tracker LLC

email: tourtracker@gmail.com

web: http://thetourtracker.com

twitter: @tourtracker

facebook: http://facebook.com/thetourtracker

[jadams602]

Ok, I'll try and clean it up a little and provide a .zip (appropriately enough ) here of the src and bin compiled version of the ANE. Might need a day or so to post it here in a cleaned up form.

[jadams602]

I should mention though that there are some AS3 ActionScript lib impls for expanding zips that you might also be able to try, but I went with the Java ANE on the assumption that it might perform a little better than an AS3 impl.

[AllanPadgett]

Very reasonable assumption. When I was thinking about this I was in fact worried about the performance and memory implications of an AS3 library.

Allan

Allan Padgett, CEO

Tour Tracker LLC

email: tourtracker@gmail.com

web: http://thetourtracker.com

twitter: @tourtracker

facebook: http://facebook.com/thetourtracker

[jadams602]

Here is a link to the src/bin of the Android ANE I whipped up to do zip expansion:

https://dl.dropboxusercontent.com/u/66764558/ZipExpand.zip

The src and ant build file are there if you want to recompile from src, updating build.properties to point to your SDKs,

or you can just use the single file build/ZipExpand.ane directly in your Android project

To use in code just do something like:

--------------------------------

import util.zip.expand.ZipExpand;

private static var zipExpander:ZipExpand;

...

// can reuse a static singleton instance of the class

if (zipExpander == null) zipExpander = new ZipExpand();

...

var destinationZip:File = File.applicationStorageDirectory.resolvePath("help.zip");

var expandToFolder:File = File.applicationStorageDirectory

zipExpander.expand( destinationZip.nativePath, expandToFolder.nativePath);

--------------------------------

expand() takes two arguments, a fullpath to the .zip and a fullpath to the directory you want the files extracted to (preserving the internal folder structure of the zip contents btw)

For example, based on my use case above, after I copied my help.zip to File.applicationStorageDirectory I just extract the contents directly there, ending up with a help/..  folder since the 'help/' prefix is contained in the zip entries.

I only implemented a simple synchronous expand() function but I'll leave it to others to play with adding an async version when an event callback if desired. The synchronous version was more than enough of an improvement in speed for me. The Java code should be the most efficient and fastest buffered io way of doing the expansion.

Enjoy and feel free to modify or reuse as needed. I compiled with swf-version="21" and tested only on android sdk versions 9 (Kindle Fire) and 18 (Google Nexus 7) so I can't guarantee no issues with any other devices.

[reinier_77]

Hi all,

I ran into the same problem on Android: File.copyToAsynch is terribly slow. For 200 files of about 42mb total, it took my Samsung Note 10.1  85 secs to copy from app dir to storage dir! (I have to copy the files to storage dir to be able to read modification date, which is not present in app dir, for an update scheme.)

The solution I ended up with is very simple:

Iterate recursively over folder that you want to copy.

For each file that is not a directory, open the file (using FileStream) , read the bytes, write the bytes to a file on the target directory and close the streams.

This took only 5 secs for the same 200 files / 42 mbs. As this happens only on first run of my app, this is very acceptable.

Here is my code :

------------------------------------------

var folder:String = "content/";//supply your folder here

var src:File = File.applicationDirectory.resolvePath(folder);

var goal:File = File.applicationStorageDirectory;

var bytes:ByteArray = new ByteArray();

var stream:FileStream = new FileStream();

var iterate:Function = function(dir:File, path:String):void

{

                                        var files:Array = dir.getDirectoryListing();

                                        for each(var file:File in files)

                                        {

                                                  if (file.isDirectory)

                                                  {

                                                            iterate(file, path + file.name + File.separator);

                                                  }

                                                  else

                                                  {

                                                            bytes.clear();

                                                            stream.open(file, FileMode.READ);

                                                            stream.readBytes(bytes);

                                                            stream.close();

                                                            var dest:File = goal.resolvePath(path+file.name);

                                                            stream.open(dest, FileMode.WRITE);

                                                            stream.writeBytes(bytes);

                                                            stream.close();

                                                  }

                                        }

}

iterate(src, folder);

------------------------------------------

This could easily be reworked to work asynch of course.

Hope this works for other people as well.

Reinier

[timoru_1219]

Hi Reinier,

possible convert your code to using "openAsync"?  openAsync you can add listener for the progress,

i have try, but failure many time. sorry, i am newbi in as3.

Thanks in advance.

[All_Bugs_Killer]

Thank you, it is X10 faster, great solution.

[AllanPadgett]

That is awesome. If not cash, maybe there is something else I can do for you at some point!

Allan Padgett, CEO

Tour Tracker LLC

email: tourtracker@gmail.com

web: http://thetourtracker.com

twitter: @tourtracker

facebook: http://facebook.com/thetourtracker