6 Replies Latest reply on May 9, 2016 8:08 AM by bachmarc

    Import of folderstructure with xmp data is very slow

    bachmarc

      Hi,

      I have a local LR archive on a local HD...

       

      Lighroom

      ...

      •   2006
        •       Jan
        •       Feb
        •       March
      • 2007
        •     Jan    
        •     Feb   
      • ...

       

      filled wit about 40k images .

      Older folder contain mostly JPGs and starting with 2011 more and more CR2 raws are found.

      I forced LR 4 to create XMP Metafiles and copied the whole folder structure to a NAS.

      I created a new empty catalog and clicked “+” in the folder section… LR opens an import dialog and tells my “importing images, keeping location”. Brilliant!

      That worked for testfolder 2016 and 2015 but it took some time, so I got greedy

      I deleted the catalog, created a new one and selected the mother node of the folderstructure to import everything…

       

      Now the trouble starts:

       

      Lightroom starts with older subfolders folders 2003, 2009 for some reason (only filled with simple jpgs) and is sooooooooo slow after initial 300 pics. When I say slow, I mean slow: Every 40-60 seconds a new pic appears…   during testimport thousands of fat raws have been imported quite fast...

      I stopped this import drama after an hour and LR pops up a message which tells that import of some videos failed.

       

      During the import drama cpu, network and disks are almost idle... I can access NAS:  it's idle, network is free from additional load...I can create IO till GBit LAN is saturated, watch videos, copy files in explorer... it is not a NAS issue...

       

      After the import was aborded an additiional try with 2015 folder slowed down as well... while it initially worked fine. After the abortion again an "import of some videos failed message"

       

      Logfiles would e nice....

       

      Any ideas? The only thing I found in internet is that an upgrade of LR from my current 4.4? to CC6 will not improve import speed ;o)

      Currently it looks buggy..

       

       

      Marc

        • 1. Re: Import of folderstructure with xmp data is very slow
          dj_paige Level 9

          The whole idea of using LR 4 to create XMP files and then re-importing 40000 images is problematic. There's no reason I can think of to do this.

           

          The better solution is to use the LR 4 catalog and then zero importing is needed. Why aren't you doing this?

          • 2. Re: Import of folderstructure with xmp data is very slow
            bachmarc Level 1

            The reason is simple: 2 Computers!

            The powerful computer is downstairs in a homeoffice. But just for importing a few new shots and tagging I don't want to go downstairs. I would like to see my family ;o)

             

            I wish to keep the meta info in xmp files and have a separate catalog on my laptop's LR.

             

            The idea is to import the SDCARD with laptop to NAS storage, delete the bad images, rate the nice ones,  tag the images, do some basic actions and leave it for a while...

             

            ... later with more time I use the better PC, with calibrated screen and GPU to fine-tune the top images...

             

            I would like to have the data in sync..

            • 3. Re: Import of folderstructure with xmp data is very slow
              dj_paige Level 9

              XMP is not the solution.

               

              Whatever you do on the laptop, you can transfer the new edits to the desktop computer via a catalog, not via importing the photos and using XMP.

               

              Import the photos into Lightroom on your laptop, perform whatever tasks you want. Then use File->Export as Catalog on the laptop, this creates a catalog file for transfer to the other computer, then on the other computer use File->Import from Another Catalog.

               

              No need to import photos more than once, even though you have two computers.

              • 4. Re: Import of folderstructure with xmp data is very slow
                bachmarc Level 1

                Yes, its a workaround because LR has no real cloud features... what I want to do is  decide freely which device I use without pushing catalogs around because everything is saved on a central  ZFS based, mirrowed NAS and frequently offline-backuped.

                 

                I want to work downstairs, show my wife upstairs and finish downstairs... or the other way around. That's the idea of keeping metadata with the files... the only local thing is the preview collection.

                 

                This options is designed in LR... technically it seems to have some bugs or bottle necks... that's what I would like to know.

                A catalog approach is not helping...

                 

                Is an online catalog an option? Or would this slow down everything? I think I read it is to slow due to DB and SAMBA

                • 5. Re: Import of folderstructure with xmp data is very slow
                  dj_paige Level 9

                   

                  This options is designed in LR... technically it seems to have some bugs or bottle necks... that's what I would like to know.

                  A catalog approach is not helping...

                   

                  The catalog approach is the safest and most complete way to transfer the information from Lightroom on one computer to Lightroom on another computer. There are no bugs or bottle necks that I know of.

                  1 person found this helpful
                  • 6. Re: Import of folderstructure with xmp data is very slow
                    bachmarc Level 1

                    Yes, but it is not what helps me ... I simply have to import some folders without having LR falling asleep ;o)

                     

                    Catalog export is o option, I want it "cloudy" and automatic