21 Replies Latest reply on Feb 25, 2018 2:04 PM by richardplondon

    Can't delete duplicates without deleting them all

    samuell18410272

      I'm starting to pull my hair on this problem. For some weird reasons, I have duplicate of many of my edited photos. The duplicate is unedited and seems to be in the catalog only, as I have no duplicate in the actual folders. I wouldn't mind deleting the duplicate in lightroom one by one but when I do, all the duplicates (including the original) go away. I just don't know what to do now.

      Capture.PNG

      (When I delete or remove any of the grey one, the same looking grey and the green go away frustratingly)

        • 1. Re: Can't delete duplicates without deleting them all
          Just Shoot Me Adobe Community Professional

          In the screen shot you posted I only see one image that has been edited.

           

          Also you are not showing the actual File Name with Extension.

           

          When was the last time you restarted your computer?

           

          A larger screen shot showing the Folders in the "Folders" section of the left hand panel might help.

           

          Have you Right Clicked on one of the suspected duplicates and selected Show in "Finder/File Explorer" to see if they are in a different folder than you think they are in?

          • 2. Re: Can't delete duplicates without deleting them all
            JohanEl54 Adobe Community Professional

            This suggests a problem in the database itself. It seems you have multiple records for one and the same image, which should not be possible. You can try to solve this as follows (make a catalog backup first!):

            1: Select the edited version of the image

            2: Go to menu 'Metadata - Save Metadata to File'

            3: Delete the image from the catalog (which should cause all copies to be deleted)

            4: Import the image again either by using 'File - Import' or by right-clicking on the folder in Lightroom and choosing 'Synchronize Folder'.

            Because you have written the metadata (i.e. the edits) to XMP before you deleted the image, the edits will be read when you import the image again. And (hopefully) now your import will only create one database record as it should.

            • 3. Re: Can't delete duplicates without deleting them all
              Just Shoot Me Adobe Community Professional

              JohanEl54  wrote

               

              This suggests a problem in the database itself. It seems you have multiple records for one and the same image, which should not be possible. You can try to solve this as follows (make a catalog backup first!):

              1: Select the edited version of the image

              2: Go to menu 'Metadata - Save Metadata to File'

              3: Delete the image from the catalog (which should cause all copies to be deleted)

              4: Import the image again either by using 'File - Import' or by right-clicking on the folder in Lightroom and choosing 'Synchronize Folder'.

              Because you have written the metadata (i.e. the edits) to XMP before you deleted the image, the edits will be read when you import the image again. And (hopefully) now your import will only create one database record as it should.

              This will notwork.

               

              OP Do Not Do this.

               

              When you Delete image files from the Catalog it also removes the XMP file for that/those images.

              You can REMOVE PHOTO and then Select the REMOVE BUTTON which should keep both the image and the corresponding XMP file in the original folders they are in.

              • 4. Re: Can't delete duplicates without deleting them all
                JohanEl54 Adobe Community Professional

                Not true. When you save metadata to files, a separate XMP file is created next to a raw file. In case of tiff or jpeg, the metadata are written into the file header. Both are NOT deleted when you delete the image from the Lightroom catalog (only the metadata in the catalog itself is deleted), and are read by Lightroom when you import the image again. Try it yourself with a single image if you don't believe me.

                • 5. Re: Can't delete duplicates without deleting them all
                  JohanEl54 Adobe Community Professional

                  BTW, when you 'delete' the image, you'll get the question whether you want to delete it from the catalog only, or also from disk. Obviously, you want to delete it from the catalog only, not from disk. I think that is what you refer to. 'Delete' and 'remove' is the same thing in this case, so it seems a matter of semantics only.

                  • 6. Re: Can't delete duplicates without deleting them all
                    JohanEl54 Adobe Community Professional

                    "When you Delete image files from the Catalog it also removes the XMP file for that/those images."

                    So just to make sure I checked this again. This statement is false. The XMP file is not deleted and so my suggestion is definitely worth trying.

                    • 7. Re: Can't delete duplicates without deleting them all
                      WobertC Adobe Community Professional

                      (I am late to the discussion)

                      I see the problem as very strange but my first suggestions would be to-

                      Delete the Previews folder.

                      Restart the computer.

                      Restart Lightroom, allow previews to re-build.

                       

                      Consider also-

                      If you save Metadata to XMP files with the intention to Import photos into a new catalog then much information is lost!

                      Many things aren’t stored in XMP, including-

                      Flags, Virtual copies, Develop History, Develop panel switch positions, Collections, Publish Services, Books, Slideshows, Saved Prints, Web Galleries, and a bunch of other bits.  (Reference: Victoria Bampton )

                      1 person found this helpful
                      • 8. Re: Can't delete duplicates without deleting them all
                        JohanEl54 Adobe Community Professional

                        My suggestion is not to import all the photos into a new catalog, my suggestion is to use this only on the offending photos (the ones with multiple thumbnails) and then import them again into the same catalog to see if that solves the problem of having multiple database entries for the same image. You are correct that this would delete a few things, such as the edit history, but nothing major. Try it with one photo first and make a catalog backup before you do.

                        1 person found this helpful
                        • 9. Re: Can't delete duplicates without deleting them all
                          Just Shoot Me Adobe Community Professional

                          Your terminology is incorrect.

                          You REMOVE images from the catalog. You Don't DELETE them.

                          There is a DELETE button (Delete from disk. Still says Delete) and a REMOVE button.

                           

                          Your instructions clearly state DELETE. If some one clicks the DELETE from Disk button they will lose the XMP file.

                           

                          DELETE and REMOVE are not the same thing.

                          • 10. Re: Can't delete duplicates without deleting them all
                            JohanEl54 Adobe Community Professional

                            Actually Lightroom uses the words Delete and Remove inconsistently. There is a 'Remove Photo' menu, and there is a 'Delete Rejected Photos' menu. Contrary to your suggestion, the latter does not act any differently than the first. When you invoke this menu while you're in a folder, Lightroom will give you the same dialog. And when you invoke this menu while you're in a collection, it doesn't even remove the rejected photos from the catalog. It will only remove them from the collection. And then there is the Delete key, wich will do the same as the 'Remove Photo' menu. So please, no semantics discussion. It is clear that the idea is that you remove/delete/clear the photo from the catalog, but obviously not from the disk.

                             

                            BTW: Lightroom has no command that will remove the photo from the catalog and delete the XMP file from disk, but not the photo from disk. Someboy might conclude that from your earlier post, where you said “When you Delete image files from the Catalog it also removes the XMP file for that/those images.” That is wrong or at least confusing terminology by your own definition given above, where you say that you REMOVE images from a catalog and DELETE them from a disk.

                             

                            So please let's stop this semantics discussion. It won't help the OP with his problem.

                            • 11. Re: Can't delete duplicates without deleting them all
                              Vienna_Sydney Level 2

                              Coming from another thread, have exactly the same problem with one folder. This is duplicated on my notebook with the identical folder. None of the pictures are, or were edited in LR, but edited in Bridge.

                               

                              This problem has existed for quite some time, but only realized when known photos were imported during synchronization. Tried importing them instead, with the same result. Removed the whole folder and reimported them, which was ok at the first import. The problem repeated itself on subsequent synchronization.

                               

                              The only difference with mine, I can delete (remove) 6 out of the 7 pictures and the last one remains. Had done this several times, only to finish up with duplicates after a while.

                              Capture.JPG

                              • 12. Re: Can't delete duplicates without deleting them all
                                Laura Shoe Adobe Community Professional

                                This method of removing from the catalog and reimporting should only attempted if  pick/reject flags, virtual copies, step-by-step edit history and membership in collections are not important to the user, since they would be lost. I realize that the grid screenshot from the OP shows that these photos have none of these, but other readers with the same issue may.

                                • 13. Re: Can't delete duplicates without deleting them all
                                  samuell18410272 Level 1

                                  Just to do a follow up on this. I have not resolved this problem yet.

                                   

                                  I kinda closed the issue with a workaround, wich was to export XMP on all my 18k+ photograph of my entire catalog. Created a new catalog and re-imported them all. The issue isn't resolve in a sense that the problematic pictures are still problematic, but at least when I "Synchonize" and tick "Open dialog box", I can tick "Do not import suspected duplicates" and then they disapear from the importing dialog box.

                                  • 14. Re: Can't delete duplicates without deleting them all
                                    JohanEl54 Adobe Community Professional

                                    Is it possible to post one of those images, including the XMP file? Perhaps in Dropbox?

                                    • 15. Re: Can't delete duplicates without deleting them all
                                      Vienna_Sydney Level 2

                                      Ran tests on several PC's, using the same folder and contained photos. All PC's exhibited the same duplicates using LR 7.2

                                       

                                      One PC has an older version as well (5.7), where the same folder was synchronized a few times. The result was quite pleasing, as no duplicates were amongst them. The number of pictures shown agreed with the ones in Windows Explorer.

                                       

                                      So I guess the problem is the current version, or any one in between 5.7 and 7.2, as the catalogues were upgraded each time.

                                       

                                      Might delete the current catalogue on the standby PC and upgrade the one from version 5.7. See what that brings to light

                                      • 16. Re: Can't delete duplicates without deleting them all
                                        Vienna_Sydney Level 2

                                        Have concluded my tests, with the following results:

                                         

                                        Opened the catalogue from LR 5.7 and synchronized it about 10 times, to make sure it received all newer pictures.

                                        The result was 10,060 pictures

                                         

                                        Opened and upgraded this one in LR 7.2 and once finished, synchronized just this folder once.

                                        The result was 10,150 pictures

                                         

                                        Recognized most of the additional photos, as these are the ones that get duplicated after every synchronization. Subsequent synchronization, don't show any more new ones

                                        • 17. Re: Can't delete duplicates without deleting them all
                                          Laura Shoe Adobe Community Professional

                                          This does sound like a bug. I would recommend posting over on Adobe's feedback site so that it gets attention from technical staff.

                                          1 person found this helpful
                                          • 18. Re: Can't delete duplicates without deleting them all
                                            Vienna_Sydney Level 2

                                            Thanks Laura, will do so

                                            • 19. Re: Can't delete duplicates without deleting them all
                                              richardplondon Level 4

                                              JohanEl54  wrote

                                               

                                              "When you Delete image files from the Catalog it also removes the XMP file for that/those images."

                                              So just to make sure I checked this again. This statement is false. The XMP file is not deleted and so my suggestion is definitely worth trying.

                                              You both may be right - but careful terminology is very important here.

                                              • When you Remove images from the Catalog (without deleting the file on disk) the XMP file is not deleted.
                                              • When you Delete images (both from the Catalog and from disk), the XMP file is deleted also. So if you restored an image file from Trash / Recycle / separate archive location, say, then the XMP file - if separate - would also need to be restored.

                                               

                                              But there are several adverse consequences from writing metadata, removing, reimporting. So IMO this is a last-resort tactic only.

                                              • All your editing History is wiped
                                              • Any membership in Collections is obliterated (and would not be reinstated) - same for Print Collections.
                                              • Any membership in Publish collections and Publish status would be obliterated; this may lead to the external copy being removed in due course, from wherever it has been published to.
                                              • Virtual Copies, also proofing copies, cannot be written out as metadata hence cannot be later reinstated this way.Their removal is final, unless restored from a Catalog, since a Catalog is the only place where these can ever live.
                                              • If you remove a Master Copy which has a Virtual Copy or Copies present too, then a Virtual Copy will become the new Master Copy so the stated operation could not be cleanly completed - unless all VCs were removed also (irretrievably).
                                              • Some other specific attributes of an image, or of an image's context, are not included when metadata is written out.
                                              • 20. Re: Can't delete duplicates without deleting them all
                                                JohanEl54 Adobe Community Professional

                                                This is all fine and true, but it would be good to read an advice in the context of the problem.

                                                • 21. Re: Can't delete duplicates without deleting them all
                                                  richardplondon Level 4

                                                  OK: my suggestion would be (assuming you haven't already tried this) to move the location of the files outside LR, and then re-connect the folder concerned using Find Missing Folder.

                                                   

                                                  That should re-build the paths information in the Catalog for the pictures concerned and in the course of doing so, I would expect spurious duplicates to get cleaned up.

                                                   

                                                  I would use a location for doing this, that is not within the user login's specific files area, but rather is located elsewhere on disk. Making sure there are no case inconsistencies / Case Inconsistencies anywhere, since those are a notorious confusion generator between the OS and LR.

                                                   

                                                  IMO fixing whatever's wrong, not trying again from scratch, is always preferable... with LR as with other programs of the same nature.

                                                   

                                                  Same as, if something's amiss with someone's employment paperwork you probably wouldn't fire them and re-hire them under a fresh employee number. Their former employee number will be involved in too many, laboriously achieved, useful records. Rather, you'll try to sort out whatever errors may have crept into their current file.