6 Replies Latest reply on Jul 14, 2014 9:22 AM by alanterra

    Very, very slow file operations

    alanterra Level 1

      I needed to make a specific change to an exif field of some images. So what I did was to move identify the files that I needed to change in Lightroom (they were in various folders on my disk) and move them to other folders so they would be easy to operate on. Then I used exiftool to change the field in these files, and then I thought it would be easy to read in the changed data and move them back to their original folders in Lightroom. What I found included:

       

      • I tried to Synchronize Metadata on the folders after changing the exif field. One folder had about 500 images in it, the other about 2,000. After 3 hours, the progress bar had barely moved, and I aborted.
      • When I tried to quit Lightroom it said that it was "writing metadata", even tho the synchronization had changed the external data and it should only have been reading metadata. I quit anyways and rebooted.
      • After rebooting, and relaunching Lightroom I looked at the "metadata status" column in Grid view. It was (as always, it seems) completely wrong about which files were up to date.
      • I was able to "read metadata" on these 2,500 images in "only" about an hour or two. Note that I didn't ask Lightroom to update the cached images, only to read the metadata. Reading metadata at 1,000-2,000 images an hour seems exceptionally slow.
      • Now I am trying to move the files back to their correct folders. I dragged about 1,000 raw images (CR2 + xmp) from one folder to another in Lightroom. Three hours later it is about 50% finished according to the progress bar. I'm going to bed and hope it will finish by the morning. I have to move all the other images as well, but because Lightroom won't update the filter bars in Grid View while moving files, I can't start these moves until Lightroom finishes the first.

       

      Do other people fine absolutely absurd speeds for file operations in Lightroom? Is there any solution other than quitting and rebooting (which doesn't always work)? I hate to quit in the middle of a synchronize, read metadata, or move files operation even when it seems to be barely chugging along, because I am afraid that when I abort the operation, I will somehow leave a file orphaned, or it's metadata out of sync (which happens to me all the time, but I never know why).

       

      When I am doing other operations I also find that I can use Lightroom for only a few hours before it gets so slow that I quit and reboot, after which operations speed up considerably. This doesn't seem like it should be necessary for a "modern" program and operating system, but in my experience it is.

       

      I'm using a Macintosh (4 core i7, 16 GB of RAM), Mavericks latest, Lightroom 5.5, and a RAID disk, so hardware shouldn't be limiting. During this last interminable copy operation, it appears that Lightroom is pegged at 100% of a processor (core).

        • 1. Re: Very, very slow file operations
          alanterra Level 1

          After posting this, I looked at the folders in question in the Finder. Lightroom is moving files (a CR2 + an xmp file pair) at a rate of about file pair every 6 seconds, or 10 images a minute. It hasn't maintained this speed tho, because I asked it to move about 1,700 images, starting 3 or 4 hours ago. (3 hours * 60 minutes * 10/minute = 1,800 images), and it is only about 2/3 of the way done according to the progress bar. This is slow!

          • 2. Re: Very, very slow file operations
            dj_paige Level 9

            This could be the correct speed to move this many images, because each time a photo is moved, Lightroom has to update its database as well.

             

            Some people recommend doing large file moves in your operating system and then reconnecting via Adobe Lightroom - Find moved or missing files and folders, because they claim it is faster (I haven't tested it myself)

            • 3. Re: Very, very slow file operations
              alanterra Level 1

              I'm not sure what the "correct" speed is, but it is excruciatingly slow, and the speed can vary considerably according to how much you have already used the database. When I posted this, it was taking over 10 seconds to move a file (1,000 files in over 3 hours). After rebooting and reopening Lightroom, my speed for copying files started out at much less than a second per file, and, then after I made extensive changes to my database the speed increased to 2.5 seconds/file moved. Again quitting and reopening Lightroom, I could bring that number down to 0.6 seconds/file. So I found at least a 4-fold increase in speed by quitting and reopening my database, and probably a speed increase of 10 to 15-fold (or more!) compared to last night when things had really bogged down.

               

              I can't imagine that Adobe will address this. The work-around is, of course, to quit Lightroom every 30 to 60 minutes, and think about rebooting after each quit. The problem is that if you start a large process without remembering to quit first, you have to abort it and then hope that you can figure out what completed and what did not after you reopen your database.

              • 4. Re: Very, very slow file operations
                dj_paige Level 9

                The workaround, as I suggested is to use your operating system and then reconnect in Lightroom.

                 

                You may also need to optimize your database, if you haven't done so recently.

                • 5. Re: Very, very slow file operations
                  alanterra Level 1

                  Sorry, but I don't think your workaround works. I am moving a bunch of files from one folder to various folders (using Lightroom to discriminate between files based on their exif/iptc values), then I change them using an outside program (exiftool), then I put them back in their original folders. This involves having Lightroom read in the new data in the files, which is what is so slow. Reconnecting, if I understand it, is for when you move a folder of files to another location, and Lightroom needs to be informed of that move.

                   

                  As I pointed out, the problem is that Lightroom gets progressively and painfully slower the more you use it. But quitting and restarting can be annoying if you are in the middle of an operation because you can lose context (especially if you restart your computer, which seems to help).

                   

                  And I do optimize my database nearly daily when using Lightroom.

                   

                  I would spend the time to fully document this problem and report it as a bug, but my experience is that Adobe shows no evidence of actually reading my bug reports, let alone responding to them.

                  • 6. Re: Very, very slow file operations
                    alanterra Level 1

                    Oh, and a final statistic. I did some rough timing of metadata operations in LR and these are my notes to myself:

                     

                    In the AM today, I did some tests, and found these speeds:

                    • After a boot and newly opened database: 0.25 seconds/file
                    • Re-opened database but no reboot: 0.6 seconds/file
                    • Database that has been used for a while with lots of changes: 2.5 seconds/file (today)
                    • Database that has been used extensively: well over 10 seconds/file (last night)

                    which is really unacceptable. The difference between the first and the fourth means that I can update 2500 photos in about 10 minutes sometimes and in 8-10 hours at other times, all depending on whether I have been using LR extensively previous to the update. The idea that I have to quit Lightroom every 30 minutes and reboot is just so depressing. Adobe does its best to destroy ones productivity.