Lightroom corrupting RAW files? Hardware issue?

[AdamCohn]

I've started having some issues with RAW files being corrupted when I open them in Lightroom. I've done a lot to troubleshoot and narrow down the symptoms, and here is what I can tell you. If anyone has any ideas for how to solve this, I'd really appreciate it!

Equipment

Canon 7D, 1 32GB CF card, 1 4GB CF card

Lenovo Laptop, Win 8; Lightroom 4

Dell PC, Win7, Lightroom 5

Here's what the corruption looks like in my Library view. Note that all of the photos are the same. I just rapid-fired a bunch of shots down the hallway to test this out.

I first noticed the corruption on the PC, pulling photos from the 32GB card. I bought a new hard drive (SSD hybrid), installed a clean version of Windows 7, and upgraded to Lightroom 5, then the 5.3 update. I re-downloaded some photos from the 32GB card, and they still show up corrupted. So here is my further testing:

• I shot a series of photos like shown above on the 32GB card, downloaded them on the laptop with a new USB cable. The photos look good.
• I shot a series of photos on the 4GB card, downloaded them to the laptop with the same cord, they look good.
• I imported the same photos photos from the 4GB card and from the 32GB card on the PC with my existing Lighrtoom Catalog and they are corrupted.
• I created a new Lightroom Catalog on the PC and downloaded the same photos from the 4GB card and from the 32GB card on the PC and they are corrupted.

So what gives? I think we can rule out corruption or issues on the hard drive, the OS, the USB cable, and both of the CF cards. It baffles me that a new HD, clean OS install, new catalog, new USB cable, and upgraded Lightroom haven't solved the problem. Is it possible that there is some other part of the hardware on my PC which is corrupting the files?

I'd love some insight if folks have any ideas!

Thanks,
Adam

[99jon]

I would suggest getting Dell or you reseller to run a utility check on the hard drive or USB ports. Definitely a hardware fault as LR does not write to CR2 files.

[AdamCohn]

Can you explain a little more what you mean when you say that Lightroom doesn't write to CR2 files? Thanks!

[JP Hess]

Your raw files remain completely untouched. When you make changes in Lightroom they are stored in the catalog or written to XMP files if you have that option selected. Nothing is ever written directly to the raw file in Lightroom, or Camera Raw if you are using Photoshop, or Photoshop itself.

[areohbee]

On laptop, you assessed "photos look good" by importing in Lightroom, and waiting until initial view is replaced by raw rendering? (or some other software which renders the raw instead of showing jpeg preview. if not, then that test doesn't mean much, except that it might take a few seconds before thumbs appear corrupt in Lightroom).

Have you tried a different card reader?

~R.

[AdamCohn]

Just to make sure, on the laptop I developed one of the photos which appears corrupt on the PC, and it looks fine in Develop mode in Lightroom as well as in the final jpeg I developed. So I think that the preview isn't the issue. In terms of card reader, I just plug the USB into the side of my camera. I've never seen a reason to add another piece of hardware into my process. Is there something I am missing there? Thanks though!

[areohbee]

AdamCohn wrote:

Is there something I am missing there?

No. If camera works then you don't need (and problem isn't) reader. But for the record: many people use readers because direct from camera either doesn't always work reliably, or it's less convenient. For example, I use reader because I have one permanently installed and available, whereas I don't keep USB cables permanently strung - 6 o' 1 half-dozen of other...

PS - I think ram problem is likely given the rest..

Standing by...

R

[Hal P Anderson]

Adam,

You say the images look good on the laptop, but they're corrupted on the PC. I assume that you're using LR to view them in both places.

So there is something in the PC that is corrupting them. I don't see why you can rule out the hard drive. New hardware fails more often than hardware that has run for a few months.

I'd check my hard drive and my RAM. Either can lead to corrupted images. Download Memtest and run it against your memory. Disk manufacturers often supply diagnostic programs for their drives. You probably can find one on their website.

Hal

[AdamCohn]

Correct, I am using Lightroom to view and develop on both computers. LR4 on the laptop, LF 5.3 on the PC.

I think that I can rule out the hard drive because the problem surfaced on my old hard drive and also is apparent on my new hard drive. I think you might be right that it is a RAM issue. I'll try out Memtest... hopefully I will be able to understand what the results tell me. I'll report back. Thanks!

[AdamCohn]

Bad news: I ran Memtest for 1 hour which was one full pass and it found no errors at all. I will run it overnight when I go to bed tonight, but this doesn't seem to be the likely culprit. Any other suggestions?

Also worth noting: when I was finding a USB stick to install Memtest on I put a stick into my PC to check what was on it. There were photos on it, including a JPG which appeared in the Windows preview with the same corruption as I showed before. I opened the JPG in the photo viewer and saw the corruption too. Note that these photos were taken a year before I first started seeing this corruption issue! So I put the same USB stick in my laptop and the same JPG looks totally fine. I'm so baffled! Is it just time for a new PC?

[areohbee]

Hmm - bum USB port?

[AdamCohn]

I've used various ports when testing this, so I don't think it's that... ugh!! It just seems like there's something wrong deep in an otherwise fine PC!

[digitaloxygen]

Was this ever figured out AdamCohn?  I have a very similar issue.  I have actually watched two photos over the last two days become corrupt while viewing them in Lightroom.  I am not suggesting LR is the cause (as I know this is almost always hardware related) but it has happened right in front of my eyes while LR is interacting with the images (I believe).

Scenario:

Images were

* Imported into LR on different days (about a week apart)

* Imported to different hard drives during import (one an SSD, another to a typical HDD)

* Behaving normally in LR until corruption (i.e. previews were built without issue, was able to develop and export images. Showed fine in Finder).

After viewing the images in Loupe view and watching them flip instantly to a visual mis-mash similar to what you showed the images were permanently screwed.  The appear the same way in Finder and in Photoshop

I have not noticed any other issues in any other app or area of the system.  Everying seems to be running smoothly.

Troubleshooting:

* Memory tests:
  - Built-in Apple memory test (normal + extended).

  - MemTest (as described here: MemTest for Mac OS X Tests your RAM). Ran it twice, all tests passed.

* I am running some HDD checks now, but with the two images being on two different drives I suspect the likelihood of this being a drive issue is extremely small.

Suggestions?

[Geoff the kiwi]

Where are the files stored?? Lightroom only gets the OS to read raw files... so the OS is giving Lightroom corrupted information hence the conclusion that it is a hardware issue.

[digitaloxygen]

As mentioned the two files in question are each stored on a different drive.  One is on an SSD and the other is a standard HDD.  Both internal.  That's what has me perplexed.  If the RAM is supposedly fine and they're stored on two separate drives (thus unlikely it's a drive issue) what other common element is left that could be the culprit?

I suppose in theory each drive could have had a problem near the same time but that seems very, very unlikely.

[Modesto_Vega]

digitaloxygen wrote:

Troubleshooting:

* Memory tests:
  - Built-in Apple memory test (normal + extended).

  - MemTest (as described here: MemTest for Mac OS X Tests your RAM). Ran it twice, all tests passed.

* I am running some HDD checks now, but with the two images being on two different drives I suspect the likelihood of this being a drive issue is extremely small.

Suggestions?

Which version of LR and OS X? Are the files raw? If so what type of file Nikon, Canon? Have you converted to DNG?

[digitaloxygen]

LR 5.7
OS X 10.9.5
Canon CR2 RAW files from a 70d.
Also have small jpeg (1920x1280) files as I shoot in RAW + jpeg.  The jpegs are fine which doesn't necessarily mean anything.

[Modesto_Vega]

Not too familiar with Canon (I use Nikon), do you have any software supplied with the camera that allows you to view CR2 RAW files? If so, what happens if you "Show the file in Finder" and try it to view it with the software supplied by Canon.

[digitaloxygen]

I could try that but I'm 99.9% sure I will get the same result.  If LR, PS, Finder and OSX Preview all show the file as being corrupt, I'm pretty sure the Canon software will too.  I'm not concerned about being able to read the corrupt files at this point (I have backups) I am much more concerned about identifying the cause of the problem to prevent it from happening again.  If hardware is failing I'm happy to replace it I just need to know what is failing.

Additional troubleshooting:
* Ran OS X "Verify Disk Permissions" and "Repair Disk Permissions" on OS drive (the SSD). Not given the option to run on the secondary HDD. Some minor permissions issues with unrelated apps (iBooks, Safari, Feedback Assistant, some others).
* Ran Verify Disk on both drives.  Status OK.

* Checked S.M.A.R.T. and other info on both drives using DriveDx (DriveDx - the most advanced drive health diagnostics and monitoring utility).
   - SSD shows overall "health" rating of Average. Lifespan shows "Good", 98%.
   - HDD shows overall "health" rating of Good.

Again just looking for some suggestions on which piece of hardware to look at next to try and identify the root cause of this corruption.

[Modesto_Vega]

digitaloxygen wrote:

I could try that but I'm 99.9% sure I will get the same result.  If LR, PS, Finder and OSX Preview all show the file as being corrupt, I'm pretty sure the Canon software will too.  I'm not concerned about being able to read the corrupt files at this point (I have backups) I am much more concerned about identifying the cause of the problem to prevent it from happening again.  If hardware is failing I'm happy to replace it I just need to know what is failing.

I think the files are corrupted. But to be 99.99% sure I would try with Canon software, you should get exactly the same result (if you don't we are looking at something seriously weird). To rule out hardware failure, I would copy the files to a different computer (if you have one) and check if you have the same problem with another computer.

If the same exact files in 2 computers look the same, your are 99.9999% sure the files are corrupted. If you can view the very same files on another computer without problems, I suggest that the cause might be some type of graphics card problem.

The next thing would be figure out how the files got corrupted in the first place. But I suggest you worry about that after copy the very same files to another computer.

[digitaloxygen]

I appreciate the kind advice Modesto but I think you're off base on the graphics card problem.  There is no way a graphics card would cause only some images to show corruption.  The graphics card has no concept of individual files. Graphics card issues would be more widespread and would not somehow limit themselves perfectly to individual files displayed in different visual locations on the screen in different applications.  Just a PSA for others reading this thread.

I also won't bother copying the corrupted files to another system.  The corruption has already happened.  100% guaranteed.  If two pieces of industry leading software show the same issue as well as multiple OS viewers there is no reason to try another.  This is not a general format or compatibility issue (which might benefit from trying another piece of software), it is clear cut corruption.

[digitaloxygen · Accepted solution]

Ah HA!  Found the issue.

I was mistaken when I said I had viewed / edited / exported one of the files earlier.  It turns out it was another file in the series.  After more testing with the original memory card it's clear the corruption happened at on the card it self.  Attempting to view the images on that memory card via the built-in card reader on my Macbook Pro as well as a 3rd party USB card reader both show the same corruption.  The card (or perhaps camera, though less likely I think) corrupted the file at capture / write time.

I believe the reason the corruption was not visible in Lightroom until I viewed them in Loupe view was that I had the "Build Previews" option in LR set to "Minimal" meaning it was using the embedded previews and not rendering a preview from the RAW file itself. Lightroom only attempted to build a standard preview from the RAW file once I opened it in Loupe view, thus that's when the corruption was first visible in LR. This is what made me think I was witnessing the file becoming corrupt on the fly. Had I set Lightroom to build a preview from the RAW file during import (using "Standard" or "1:1") the corruption likely would have shown right away.

Appreciate the sounding board and advice.  Apologies for not doing my due diligence ahead of time and ruling out the memory card.  I had been mistaken about my previous interactions with the files which threw me off path a little during troubleshooting.

Carry on, nothing to see here

[Modesto_Vega]

Great you found what caused the problem!! Never make any assumptions

[Geoff the kiwi]

Great tracking and response... thanks for reporting back with your findings..

[craigs48531685]

Hello, I just came across this discussion and am having the exact same issue. The difference is, I downloaded the files from the camera to the PC last July and have worked on them, on and off, up until a couple months ago. I have only used Adobe Camera Raw and PS in the past. I loaded up Lightroom 5 a couple weeks ago. Yesterday, I pulled those same files up in LR5 and am having the same issue that your RAW files had. Now, when I try to look at them in PS, they are corrupted there as well. Possibly my drive is going bad, but it seems very coincidental that I had no issue until I used LR. Again, these are files that I've worked on without issue before. Have you had any additional problems?