Can't delete duplicates without deleting them all

[samuell18410272]

I'm starting to pull my hair on this problem. For some weird reasons, I have duplicate of many of my edited photos. The duplicate is unedited and seems to be in the catalog only, as I have no duplicate in the actual folders. I wouldn't mind deleting the duplicate in lightroom one by one but when I do, all the duplicates (including the original) go away. I just don't know what to do now.

(When I delete or remove any of the grey one, the same looking grey and the green go away frustratingly)

[Just Shoot Me]

In the screen shot you posted I only see one image that has been edited.

Also you are not showing the actual File Name with Extension.

When was the last time you restarted your computer?

A larger screen shot showing the Folders in the "Folders" section of the left hand panel might help.

Have you Right Clicked on one of the suspected duplicates and selected Show in "Finder/File Explorer" to see if they are in a different folder than you think they are in?

[JohanElzenga]

This suggests a problem in the database itself. It seems you have multiple records for one and the same image, which should not be possible. You can try to solve this as follows (make a catalog backup first!):

1: Select the edited version of the image

2: Go to menu 'Metadata - Save Metadata to File'

3: Delete the image from the catalog (which should cause all copies to be deleted)

4: Import the image again either by using 'File - Import' or by right-clicking on the folder in Lightroom and choosing 'Synchronize Folder'.

Because you have written the metadata (i.e. the edits) to XMP before you deleted the image, the edits will be read when you import the image again. And (hopefully) now your import will only create one database record as it should.

[Just Shoot Me]

JohanEl54  wrote

This suggests a problem in the database itself. It seems you have multiple records for one and the same image, which should not be possible. You can try to solve this as follows (make a catalog backup first!):

1: Select the edited version of the image

2: Go to menu 'Metadata - Save Metadata to File'

3: Delete the image from the catalog (which should cause all copies to be deleted)

4: Import the image again either by using 'File - Import' or by right-clicking on the folder in Lightroom and choosing 'Synchronize Folder'.

Because you have written the metadata (i.e. the edits) to XMP before you deleted the image, the edits will be read when you import the image again. And (hopefully) now your import will only create one database record as it should.

This will notwork.

OP Do Not Do this.

When you Delete image files from the Catalog it also removes the XMP file for that/those images.

You can REMOVE PHOTO and then Select the REMOVE BUTTON which should keep both the image and the corresponding XMP file in the original folders they are in.

[JohanElzenga]

Not true. When you save metadata to files, a separate XMP file is created next to a raw file. In case of tiff or jpeg, the metadata are written into the file header. Both are NOT deleted when you delete the image from the Lightroom catalog (only the metadata in the catalog itself is deleted), and are read by Lightroom when you import the image again. Try it yourself with a single image if you don't believe me.

[JohanElzenga]

BTW, when you 'delete' the image, you'll get the question whether you want to delete it from the catalog only, or also from disk. Obviously, you want to delete it from the catalog only, not from disk. I think that is what you refer to. 'Delete' and 'remove' is the same thing in this case, so it seems a matter of semantics only.

[JohanElzenga]

"When you Delete image files from the Catalog it also removes the XMP file for that/those images."

So just to make sure I checked this again. This statement is false. The XMP file is not deleted and so my suggestion is definitely worth trying.

[Just Shoot Me]

Your terminology is incorrect.

You REMOVE images from the catalog. You Don't DELETE them.

There is a DELETE button (Delete from disk. Still says Delete) and a REMOVE button.

Your instructions clearly state DELETE. If some one clicks the DELETE from Disk button they will lose the XMP file.

DELETE and REMOVE are not the same thing.

[JohanElzenga]

Actually Lightroom uses the words Delete and Remove inconsistently. There is a 'Remove Photo' menu, and there is a 'Delete Rejected Photos' menu. Contrary to your suggestion, the latter does not act any differently than the first. When you invoke this menu while you're in a folder, Lightroom will give you the same dialog. And when you invoke this menu while you're in a collection, it doesn't even remove the rejected photos from the catalog. It will only remove them from the collection. And then there is the Delete key, wich will do the same as the 'Remove Photo' menu. So please, no semantics discussion. It is clear that the idea is that you remove/delete/clear the photo from the catalog, but obviously not from the disk.

BTW: Lightroom has no command that will remove the photo from the catalog and delete the XMP file from disk, but not the photo from disk. Someboy might conclude that from your earlier post, where you said “When you Delete image files from the Catalog it also removes the XMP file for that/those images.” That is wrong or at least confusing terminology by your own definition given above, where you say that you REMOVE images from a catalog and DELETE them from a disk.

So please let's stop this semantics discussion. It won't help the OP with his problem.

[richardplondon]

JohanEl54  wrote

"When you Delete image files from the Catalog it also removes the XMP file for that/those images."

So just to make sure I checked this again. This statement is false. The XMP file is not deleted and so my suggestion is definitely worth trying.

You both may be right - but careful terminology is very important here.

• When you Remove images from the Catalog (without deleting the file on disk) the XMP file is not deleted.
• When you Delete images (both from the Catalog and from disk), the XMP file is deleted also. So if you restored an image file from Trash / Recycle / separate archive location, say, then the XMP file - if separate - would also need to be restored.

But there are several adverse consequences from writing metadata, removing, reimporting. So IMO this is a last-resort tactic only.

• All your editing History is wiped
• Any membership in Collections is obliterated (and would not be reinstated) - same for Print Collections.
• Any membership in Publish collections and Publish status would be obliterated; this may lead to the external copy being removed in due course, from wherever it has been published to.
• Virtual Copies, also proofing copies, cannot be written out as metadata hence cannot be later reinstated this way.Their removal is final, unless restored from a Catalog, since a Catalog is the only place where these can ever live.
• If you remove a Master Copy which has a Virtual Copy or Copies present too, then a Virtual Copy will become the new Master Copy so the stated operation could not be cleanly completed - unless all VCs were removed also (irretrievably).
• Some other specific attributes of an image, or of an image's context, are not included when metadata is written out.

[JohanElzenga]

This is all fine and true, but it would be good to read an advice in the context of the problem.

[richardplondon]

OK: my suggestion would be (assuming you haven't already tried this) to move the location of the files outside LR, and then re-connect the folder concerned using Find Missing Folder.

That should re-build the paths information in the Catalog for the pictures concerned and in the course of doing so, I would expect spurious duplicates to get cleaned up.

I would use a location for doing this, that is not within the user login's specific files area, but rather is located elsewhere on disk. Making sure there are no case inconsistencies / Case Inconsistencies anywhere, since those are a notorious confusion generator between the OS and LR.

IMO fixing whatever's wrong, not trying again from scratch, is always preferable... with LR as with other programs of the same nature.

Same as, if something's amiss with someone's employment paperwork you probably wouldn't fire them and re-hire them under a fresh employee number. Their former employee number will be involved in too many, laboriously achieved, useful records. Rather, you'll try to sort out whatever errors may have crept into their current file.

[Laura Shoe]

This method of removing from the catalog and reimporting should only attempted if  pick/reject flags, virtual copies, step-by-step edit history and membership in collections are not important to the user, since they would be lost. I realize that the grid screenshot from the OP shows that these photos have none of these, but other readers with the same issue may.

[Vienna_Sydney]

Ran tests on several PC's, using the same folder and contained photos. All PC's exhibited the same duplicates using LR 7.2

One PC has an older version as well (5.7), where the same folder was synchronized a few times. The result was quite pleasing, as no duplicates were amongst them. The number of pictures shown agreed with the ones in Windows Explorer.

So I guess the problem is the current version, or any one in between 5.7 and 7.2, as the catalogues were upgraded each time.

Might delete the current catalogue on the standby PC and upgrade the one from version 5.7. See what that brings to light

[Vienna_Sydney]

Have concluded my tests, with the following results:

Opened the catalogue from LR 5.7 and synchronized it about 10 times, to make sure it received all newer pictures.

The result was 10,060 pictures

Opened and upgraded this one in LR 7.2 and once finished, synchronized just this folder once.

The result was 10,150 pictures

Recognized most of the additional photos, as these are the ones that get duplicated after every synchronization. Subsequent synchronization, don't show any more new ones

[Laura Shoe]

This does sound like a bug. I would recommend posting over on Adobe's feedback site so that it gets attention from technical staff.

[Vienna_Sydney]

Thanks Laura, will do so

[Rob_Cullen]

(I am late to the discussion)

I see the problem as very strange but my first suggestions would be to-

Delete the Previews folder.

Restart the computer.

Restart Lightroom, allow previews to re-build.

Consider also-

If you save Metadata to XMP files with the intention to Import photos into a new catalog then much information is lost!

Many things aren’t stored in XMP, including-

Flags, Virtual copies, Develop History, Develop panel switch positions, Collections, Publish Services, Books, Slideshows, Saved Prints, Web Galleries, and a bunch of other bits.  (Reference: Victoria Bampton )

[JohanElzenga]

My suggestion is not to import all the photos into a new catalog, my suggestion is to use this only on the offending photos (the ones with multiple thumbnails) and then import them again into the same catalog to see if that solves the problem of having multiple database entries for the same image. You are correct that this would delete a few things, such as the edit history, but nothing major. Try it with one photo first and make a catalog backup before you do.

[Vienna_Sydney]

Coming from another thread, have exactly the same problem with one folder. This is duplicated on my notebook with the identical folder. None of the pictures are, or were edited in LR, but edited in Bridge.

This problem has existed for quite some time, but only realized when known photos were imported during synchronization. Tried importing them instead, with the same result. Removed the whole folder and reimported them, which was ok at the first import. The problem repeated itself on subsequent synchronization.

The only difference with mine, I can delete (remove) 6 out of the 7 pictures and the last one remains. Had done this several times, only to finish up with duplicates after a while.

[samuell18410272]

Just to do a follow up on this. I have not resolved this problem yet.

I kinda closed the issue with a workaround, wich was to export XMP on all my 18k+ photograph of my entire catalog. Created a new catalog and re-imported them all. The issue isn't resolve in a sense that the problematic pictures are still problematic, but at least when I "Synchonize" and tick "Open dialog box", I can tick "Do not import suspected duplicates" and then they disapear from the importing dialog box.

[JohanElzenga]

Is it possible to post one of those images, including the XMP file? Perhaps in Dropbox?