4 Replies Latest reply on Jul 31, 2015 8:42 AM by pwfasprod

    Why are some duplicate images not found when importing?

    pwfasprod Level 1

      I have a catalog with roughly 70,000 images, not video. Someone sent me a flash drive with several thousand more images. When I imported them to Lightroom, it imported photographs already in my collection (yes, exclude suspected duplicates was selected).

       

      2015 release; Build 1018573

       

      • I have added metadata to those in the collection; is it possible that the metadata date is different so it saw them as new?
      • Does Lightroom not use the documentId or originalDocumentId to identify duplicates?
      • Is it a bug?

       

      I appreciate any help or suggestions in preventing this from occurring in the future.

        • 1. Re: Why are some duplicate images not found when importing?
          dj_paige Level 9

          It might be a bug, this is a common complaint.

           

          But in order for a file to be considered a duplicate, it must have the exact same file name, capture date and time, and file size. If someone sent you a flash drive, and the files had been modified before you received them, they might not meet the criteria of a duplicate (i.e. the file name or capture date or file size has changed)

          • 2. Re: Why are some duplicate images not found when importing?
            pwfasprod Level 1

            Thank you for your response—it would be great if an engineer could take a look at this because it is very counter-productive and frustrating.

            ...in order for a file to be considered a duplicate, it must have the exact same file name, capture date and time, and file size. If someone sent you a flash drive, and the files had been modified before you received them, they might not meet the criteria of a duplicate (i.e. the file name or capture date or file size has changed)

            1. File names are the same but that can be a problem in true de-duplication. LR has tables which store documentID and originalDocumentId; why not use those in case file name has changed?
            2. Capture date will not change unless they have manually done this or exported, so it it very unlikely that this has taken place.
            3. For the most part, these are unmodified originals. The file size will not have changed.

             

            I think it is a bug, too.

            • 3. Re: Why are some duplicate images not found when importing?
              trshaner Adobe Community Professional & MVP

              pwfasprod wrote:

              • I have added metadata to those in the collection; is it possible that the metadata date is different so it saw them as new?
              • Does Lightroom not use the documentId or originalDocumentId to identify duplicates?
              • Is it a bug?

               

              It's not a bug if you have 'Include Develop settings in metadata inside JPEG, TIFF, PNG and PSD files' checked in Catalog Settings?

               

              If so when changes are saved to the file (CTRL +S) the file's metadata no longer matches the original file. When reimporting the original file without metadata changes it is considered a "new file." if it is being imported into the same folder as the currently imported file it is renamed with a -2 designation.

               

              I've never used it, but this duplicate finder LR plugin may help to locate the reimported dupes:

              http://www.lightroom-plugins.com/DupesIndex.php

              • 4. Re: Why are some duplicate images not found when importing?
                pwfasprod Level 1

                You are correct, but it is very annoying. The crop information will not show in bridge or other apps if it is not saved to the file. It seems that the import dialogue would look at the date created instead of the date modified. A metadata save is not the same as cropping it in Photoshop, deleting the pixels and saving it; the original is still there if the changes are reverted. I really appreciate your response and I should have replied to you sooner but it has been a long couple of months!