Lightroom thumbnails - post disk failure

[Zone_99]

Hi,

After a catastrophic disk failure (double-failure actually), I lost a bunch of pictures in my archives.  I'm OK with losing most of them (hundreds of soccer/basketball/softball pics of my kids and their teammates.  I have most of the pictures I want of my kids but we've lost track of most of the rest of the kids) but this has left me with around 30,000 'missing photographs' in my catalog.

Most of these have blank thumbnails.  Some do not.  I thought that the catalog retains a thumbnail in the catalog and 1:1 previews are built on import or when working with a picture.

I didn't lose the catalog but the source pictures. 

So I have a lot of this:

I'm looking for recommendations on what to do here.  Do I:

• remove them from the catalog?
• find any way to determine something about the picture to see if I have it elsewhere and the catalog just doesn't reflect it?
• compare against existing pictures?
• Other suggestions?

I wish Lightroom had the ability to open two catalogs at once.  As part of the recovery process, I've imported recovered pictures from the damaged disks (THAT was a fun process) and would love to do a catalog to catalog compare so I could see duplicates and such but no luck there.

Open to ideas....

Thanks!

[dj_paige]

Step 1 should be restore the photos from their backups, to the original locations.

If for some reason (I can't imagine what) you don't have backups of your photos, I'm not sure anything will fix this problem. I don't see how any of your ideas will work in the absence of your original photos.

[Abambo]

if you have previews built, you can recover those. they will not replace the original pictures, but it will be better then nothing. Script to extract previews of lost or deleted photos

Suggestion: use 2 computers for comparing.

I suppose, that in the future, you will use more robust backup systems like a NAS with RAID.

[Zone_99]

Thanks for the script, Abambo.  I'll check it out.

For the record, it was NAS with RAID.  I had a drive failure and the RAID failed and my offsite back ups were...compromised...as explained above.

[Abambo]

You’re making me think about my own backup strategy!b

[Zone_99]

I just tried it and it works for previews I can see but not for others. Thanks for the suggestion though.

[dj_paige]

Yes of course, the previews for the other photos don't exist.

[Zone_99]

As I found out!   I wasn't sure if the script was recovering them from somewhere in the catalog or something.  Worth a shot.

[Zone_99]

@dj_paige - thanks for the quick response.

I *had* backups of my photos.  I was backing up and storing them at my parents house.  Unfortunately, my father, who has Alzheimers, threw some of them out!  That would be the double fault part of my data loss! 

After the disk failure, I did the following:

• restored from the backups I did have - which was a good portion
• used a disk utility to recover data off my old NAS drives (the failed drive)

The problem with the recovered data is that the file information is skewed.  So where the backup photos have the "Picture Catalog\YYYY-MM-DD\<picturename>.NEF" format, the recovered data may just say "<picturename>.NEF". 

For instance, in my catalog, I have the photos taken on 5/18/2013.  In the recovered data, I have the folder "2013-05-18".  The pictures in the recovered data are the ones taken on that date.  And they are named "prom-2013.NEF".

However, there is not a file called "prom-2013.NEF" anywhere in my catalog.  Which I find confusing!  If anything, I should be showing an entry in the catalog of that name with no picture associated.

[Zone_99]

Ok, there seems to be a missing workflow here.

For the pictures I have and are showing up the catalog as "missing".  The problem is the folder is also not in folder structure so there's no 'missing folder' icon in the Folder panel in LR.

The folders are in the OLD directory structure I was able to recover from back up or the damaged disk.

I can:

1. Import the pictures again - downside is that this doesn't link them to the edits that are in the catalog. That means I have to link 1 by 1.
2. Move the folder from the old structure to the new.  But again, LR has no idea about this folder until I link the first missing photograph.  Back to linking 1 by 1.
3. Adobe could fix the "Find nearby missing photos" when I select one.....  that feature doesn't work at all.

I want to:  identify the missing photos by metadata, point them to the old folder, have them link up and then use the Folder mechanism in LR to move them over to the NAS.....

Hmmmm

[dj_paige]

Zone_99  wrote

Ok, there seems to be a missing workflow here.

For the pictures I have and are showing up the catalog as "missing".  The problem is the folder is also not in folder structure so there's no 'missing folder' icon in the Folder panel in LR.

The folders are in the OLD directory structure I was able to recover from back up or the damaged disk.

I can:

1. Import the pictures again - downside is that this doesn't link them to the edits that are in the catalog. That means I have to link 1 by 1.
2. Move the folder from the old structure to the new.  But again, LR has no idea about this folder until I link the first missing photograph.  Back to linking 1 by 1.
3. Adobe could fix the "Find nearby missing photos" when I select one.....  that feature doesn't work at all.

I want to:  identify the missing photos by metadata, point them to the old folder, have them link up and then use the Folder mechanism in LR to move them over to the NAS.....

Hmmmm

Wait ... at one point you had no backups and were trying to obtain the thumbnails. Now it sounds like you have the photos? Could you clarify this?

If you have the photos, importing them again IS 100% WRONG. DO NOT IMPORT. If you import the photos, then you CAN NOT link the photos 1 by 1.

You want the photos in the exact same folder structure. Then you can re-connect (in Lightroom) entire folders or an entire hierarchy of folders in one operation, AND NO IMPORTING IS NEEDED, IMPORTING IS THE WRONG THING TO DO.

[Zone_99]

dj_paige​  - sure.  Let me see if I can clear it up. 

• NAS crash and lose the RAID
• I have some backups from about 2001 to 2012.  2013, 2014 and some of 2015 are effectively lost. 
• Using a fairly industrial recovery program, I'm able to recover files from remaining two of the three NAS disks - THAT was a learning experience!
• This gives me two things:  1) some of the pictures recovered in their directory structure although the names of the files do not always match what's in the catalog for the names.  2) a very large directory "Extra Found Files\Nikon Raw Images" with file names like:  "40416.NEF"

In the case of #1, the structure is right, the name is usually wrong (not always).  When I run "Library-> Find all missing photos", I can go in the metadata view and see what Lightroom thinks is missing and see all the Catalog information, edits, ratings and such.

So, for #1, I don't import these.  "Find missing folder" isn't going to work because file names are different so it's a 1:1 link or, in the cases where I was lucky, "Find missing folder" did work because all the right file structure data matched. So at this point, for all the pictures that I could recover with folder structure and names, I'm done.   Now I'm down to the oddities where I have to 1:1 link to retain edits and change names.

For #2, this is where the fun begins.  The files have their metadata including the last date modified which equals capture time.  So, if I look at the files with a date of 5/4/2013 and then look in the catalog for the files missing at that date, it's pretty much a match on number of files and content. 

EXCEPT the catalog is missing the thumbnails for most pictures in that state.  There's only one or two, the rest are grey boxes as shown above.   This means that I can't look at the thumbnails and go: "Hmm..bunch of soccer pictures of kids I don't know anymore, I'm OK losing them, remove them from the catalog". 

I thought LR cached a small thumbnail with the catalog, not just the 1:1 preview.  That's why I was asking if there was a way to recover them.  I could see if they were worth the bother of doing this.

The upside of #2's situation is that I can import those and put them into my normal structure.  The downside:  I lose the edits.

Does that help?

[dj_paige]

I find it somewhat irritating that you expected advice from me (or someone) without mentioning key details like you used a recovery program, which has changed the file names. How could I provide valid advice without knowing that?

Now that you have recovered the photos and the capture date and time is intact, you can match the photos with the blank gray thumbnail in LR using the capture date and time.

[Zone_99]

dj_paige​

My apologies. I am not trying to irritate anyone but simply trying to understand what is and isn't possible.  The whole situation has left me with multiple challenges to resolve.  Capture date and time is intact however, that's not all there is to file metadata and it doesn't take too much to throw something off. 

As you can see, this:

is not this:

404491 is the first file in the directory structure and lptg2013-week18-1.NEF is the first file in the missing photographs for that date.

This is why I was asking about the missing thumbnails to help me rip through this process and see if it was worth playing the match game to link them.  If they weren't worth keeping, then I wouldn't bother trying. 

My original question was about just that and trying to understand why LR had SOME of the thumbnails but not all of them.

I did, however, mention that I used a recovery program in my first response to you.

@dj_paige - thanks for the quick response.

I *had* backups of my photos.  I was backing up and storing them at my parents house.  Unfortunately, my father, who has Alzheimers, threw some of them out!  That would be the double fault part of my data loss!

After the disk failure, I did the following:

• restored from the backups I did have - which was a good portion
• used a disk utility to recover data off my old NAS drives (the failed drive)

Thanks.