Regarding the batch mode. You're import uses a session object as a connection to the repository. When you have called ".setBatchMode(true);" on your asset, any changes are not directly persisted on the session, but it's up to you to call "session.save()". This allows you to control the amount and the size of the transaction. As a transaction always has a certain overhead, it might be more performant to reduce the total number of transactions (= session.save() calls) and to increase the amount of work persisted with each transaction.
You need to experiment a little bit with the amount of changes for each transaction. If you try to put too much changes into the transaction, it might blow up your JVM, as the changes to be saved are stored temporary in memory, and you might run into an Out-Of-Memory. My personal recommendation would be call every 10 assets the session.save() or so.
Regarding the other questions: to reduce the time for the import itself it's best to temporarily disable the DAM asset update workflow; after the import re-enable it and touch all imported assets.
Performance: Run a profiler to find out the bottleneck. Optimizing the transaction size might be a big factor here.
how about using webdav and push your images using webdav client
Initially we though of using Webdav but we need to do some process like creating subassets which is not possible through webDav.
I will try out batch saving option.
I have few more doubts.
1. Today I tried for moving aroung 5k images through the scheduler in my local machine. It went smoothly with out much issues but as usual some of the renditions are failing for some of the sub assets. This is about 3 for every 10 images. Renditions are generating properly for all assets it is failing in case of sub assets only. Along with this I am getting version exceptions. Below are errors which I am facing very frequently during migration dry run. This seems to be beacuse of the non synchronized execute methods in WF steps. What is your call on these issues how can we avoid these kind of errors.
20.02.2013 19:16:00.082 *ERROR* [JobHandler: /etc/workflow/instances/2013-02-20/model_1361367956960012000:/content/dam/<IMAGE PATH>/subassets/<Image Name>/jcr:content/renditions/original] com.day.cq.dam.core.impl.AssetImpl setRendition: cannot set new rendition [cq5dam.thumbnail.140.100.png] for asset [<IMAGE PATH>/subassets/<Image Name>]: javax.jcr.version.VersionException: Unable to perform operation. Node is checked-in.
20.02.2013 19:17:01.660 *ERROR* [JobHandler: /etc/workflow/instances/2013-02-20/model_1361368020884400000:/content/dam/<IMAGE PATH>/jcr:content/metadata] com.day.cq.dam.core.impl.handler.xmp.NCommXMPHandler Stack Trace: java.lang.Exception: Unable to create revision.
Caused by: javax.jcr.RepositoryException: Unable to update item: item.save()
Caused by: javax.jcr.InvalidItemStateException: Item cannot be saved because it has been deleted externally: item.save()
20.02.2013 19:19:08.666 *WARN* [JobHandler: /etc/workflow/instances/2013-02-20/model_1361368144852589000:/content/dam/<IMAGE PATH>/subassets/<Image Name>/jcr:content/renditions/original] com.day.cq.dam.core.process.ExtractMetadataProcess unexpected error occurred during metadata extraction. Cause: Unable to perform operation. Node is checked-in. javax.jcr.version.VersionException: Unable to perform operation. Node is checked-in.
2. Can you share about experience in DAM migrations. What is the avg time it takes for migrating about 1 to 1.2 lac images which are aroung 60 gig. We thought of running the scheduler (this.scheduler.addPeriodicJob(jobName, job, config, period, canRunConcurrently);)for every 0.5 hr in that case it will go on for 3 days.