-
1. Re: File corruption after import
dj_paige Oct 30, 2013 9:13 AM (in response to drtom)This is almost always a hardware problem. If it happens weeks after the photo was imported into Lightroom, then there is a good chance that your hard disk is experiencing problems. I suggest you run some diagnostics on it.
Also, make sure you have backups of these photos and backups of your catalog file on another physical disk (not a different partition of the same disk); consider purchasing a new disk.
-
2. Re: File corruption after import
drtom Oct 30, 2013 10:38 AM (in response to dj_paige)Thank you for your input. I have two separate hard drives on my system. The OS runs on a SSD, then I have the main files on the HDD. I have tested the raw images on both drives with the same results. Also, when I use another program on the same drive, the problem does not occur. Wouldn't that suggest a problem with LR rather than the drive?
-
3. Re: File corruption after import
dj_paige Oct 30, 2013 10:56 AM (in response to drtom)Lightroom reads the photo data from the hard drive. Lightroom does not modify the photo portion of your photo files on your hard drive. So, if the photo data shows up corrupted, then it is the hard drive that is sending corrupted data to Lightroom.
(Of course, it is also possible that the data arrived on the hard drive from the camera card corrupted, but since you said the corruption sometimes happens weeks after import, its hard to claim that the data arrived at the hard drive corrupted)
-
4. Re: File corruption after import
drtom Oct 30, 2013 11:12 AM (in response to dj_paige)I am in the process of scanning my hard drives now. I will report back.
-
5. Re: File corruption after import
Geoff the kiwi Oct 30, 2013 4:28 PM (in response to drtom)What Fastone uses is the jpeg preview in the file not the raw data hence the clean look. Lightroom reads the preview first then the raw data.
It could well be that this occurred during d/load as well. How did you d/load??
-
6. Re: File corruption after import
DdeGannes Oct 30, 2013 4:42 PM (in response to drtom)I would also check your RAM at the same time. RAM modules can also cause these problems. I have had to change "flacky" ram on my computer twice.
-
7. Re: File corruption after import
drtom Oct 30, 2013 8:23 PM (in response to DdeGannes)I have completed the scans on both drives. They both check out fine, no problems found. I also did a mem check of the RAM, also fine, no problems.
-
8. Re: File corruption after import
areohbee Oct 30, 2013 8:38 PM (in response to drtom)Are you 100% certain the files were not corrupted upon transfer? I mean, if disks are really OK, and ram, then usually this problem occurred upon transfer, e.g. bad card reader (or card), or cable, or usb port, ...
Reminder: viewing jpeg preview, e.g. Fastone, doesn't count (has no bearing on whether raw is corrupt).
What is the filename extension of corrupt files?
Rob
-
9. Re: File corruption after import
areohbee Oct 30, 2013 8:48 PM (in response to dj_paige)dj_paige wrote:
Lightroom reads the photo data from the hard drive.
More specifically, the OS reads it from the drive, and Lightroom reads it from RAM - thus bad ram can also be a culprit...
Put another way: ram is the buffer between hard drive and Lightroom, so ram failure can mean bad data in Lightroom even if hard drive is fine.
-
10. Re: File corruption after import
drtom Oct 31, 2013 7:09 AM (in response to areohbee)Hi All,
I have done some more testing.
I used the same card reader to transfer multiple "good" and "bad" files to the computer. Then I imported them into LR. Same results. The files that were good previously remained good, the "bad" files remained bad.
I repeated this many times. Same results.
Next I hooked up the camera to the computer directly with the factory USB cord. All images transferred and imported into LR were fine.
Repeated with card reader. Got same errors. Then on subsequent tests with card reader, all images appear fine in LR.
So, is the culprit the card reader? I am guessing that LR, the Hard drive, Ram, and the Camera and SD card are all fine. Not sure if there is anything left to test. Wish I hadn't permnanently deleted some of the "bad" images before more testing. Lesson learned.
-
11. Re: File corruption after import
trshaner Oct 31, 2013 8:38 AM (in response to drtom)As has already been mentioned LR uses the camera file's embedded preview for display until Standard or 1:1 Previews are built. This would explain what you saw, "You will be looking at the filmstrip, and actually watch them be redrawn with the error." The initial image is of the smaller embedded preview. As the larger Standard or 1:1 previews are built using raw data the corrupted images are revealed.
SUGGESTION
When importing images select 'Build Previews' as Standard or 1:1. Leave the images on the memory card until 1) Preview Building has completed in the progress bar and 2) you've reviewed each image file at least once in the Library module's Loupe view mode. I would also suggest keeping the images on the memory card until you've done initial editing on all of the images. This will require using additional memory cards, which I explained in the below post is very cheap insurance:
Would you spend 16 cents for “temporary backup” of 1,000 perhaps irreplaceable images?
Try your card reader again using a new memory card before condemning it and also try plugging it into a different port. The culprit may actually be a failing memory card (or USB port), which simply works better using you camera's memory card controller. This will help you determine the defective component(s).
-
12. Re: File corruption after import
drtom Oct 31, 2013 11:09 AM (in response to trshaner)Thanks for the troubleshooting and great advice.
-
13. Re: File corruption after import
batjake Nov 15, 2013 8:56 PM (in response to drtom)We initially had trouble with the 7D. A few weeks ago - the 5Dii is having the same issue. We have started shooting RAW+Jpg as a backup. Glad we did. The two 30D's haven't had any trouble at all. Pretty much the last thing I want to see is a beautiful photo that my wife took of a bride with a big pink block going through the side of her head.
We're using high speed, high quality cards. Bought new ones when the trouble started but got the same results (both Lexar and Sandisk). We use an Extreme USB 2.0 CF reader. My guess is that the reader is the culprit (by the way, nobody seems to have a real answer - just guesses. It's annoying ). I've recovered the same corrupt, deleted files using the software that came with one of the cards. The ones that were giving me trouble turned out fine. Also no issues so far if I copy/paste with the camera actually hooked up to the computer.
Planning on upgrading to the Lexar Professional USB 3.0 Dual-Slot Reader (UDMA 7) and not deleting anything until we review each file. For some reason, the huge 17-22mpix CR2 files are fragile.
Obvious advice: Be careful...and good luck...
-
14. Re: File corruption after import
trshaner Nov 16, 2013 7:37 AM (in response to batjake)If you're on a Windows system you can test your memory card reader and cards using the below test program. I downloaded and tested the utility and it works well on my Windows 7 system. There are similar utilities available for Mac OS X, but I'm only running Windows so can't recommend specific ones.
http://www.ricksdailytips.com/flash-drive-card-tester/
Testing flash memory devices like CF, SD, and SSD can be confusing. All of these devices have the ability to reallocate bad sectors when detected. They also use wear-leveling, which means cards that appear good may over time exhibit errors. Wear-leveling attempts to control the number of memory cell read/write cycles by "distributing" them across the card's full storage-space. In a typical shoot you probably use less than the memory card's full capacity, but the card's wear-leveling algorithm is constantly reassigning different previously unused memory sectors. If any memory cells in that “previously unused” storage space are defective you may not experience file corruption until you’ve used the memory card numerous time.
Flash memory devices also have the ability to map out and "reallocate" bad sectors using "spare blocks" reserved for this purpose. This means the next time you use the card it may work fine. This is also one of the reasons why it is impossible to test a memory card with 100% assurance, since you can't physically access and test the cells in these "spare blocks."
That said you still can run the above flash memory test utility, which will write, read, and compare all of the card's "accessible" memory cells for errors. I suggest running this test utility (or a similar diagnostic program) with every new memory card you purchase or those you suspect have issues. It's also a good test of your memory card reader. If you have more than one memory card that exhibits errors then your memory card reader is probably the cause. Try testing the same cards using a different card reader.
WARNING: Prior to testing make sure you remove and backup all image files that are on the memory card. This test utility erases all data on the card! Also it is a good idea to reformat all memory cards inside your camera that you have tested with this utility or any other diagnsotic program.
This Flash Memory Guide from Kingston has information on flash memory devices applicable to most all manufacturers' devices.
-
15. Re: File corruption after import
trshaner Nov 16, 2013 3:12 PM (in response to trshaner)Just thought this might be of interest.
I just had an opportunity to test a new SanDisk Extreme 32GB 120MB/s UDMA7 card received today. Using the Flash Drive Test Utility v1.14 (see above link) 11 errors were detected. I went back and rechecked SanDisk Extreme 16GB, 8GB, and 4GB cards that tested good when new. They all tested good indicating the 32GB card obviously has some issues. I ran the test two more times and got errors at the exact same 11 LBN addresses, so they are hard unrecoverable errors. The card is being being returned for a replacement! I'll report test results for the replacment card when recieved.
-
16. Re: File corruption after import
trshaner Dec 3, 2013 2:23 AM (in response to trshaner)Update On 32GB CF Card Errors
I received a replacement SanDisk Extreme 32GB 120MB/s UDMA7 card from B&H and it exhibits 11 Fatal Read Errors at the exact same LBN locations as the first card using Flash Drive Test Utility v1.14. It appears this test utility is not compatible with 32GB UDMA7 cards. I sent an email to SanDisk Tech Support and received two responses:
I would like to inform you that the software that you are using is not a suggested software to test our cards so I would recommend you to us the card in the device that you have. If you face any issues while you are using this card on your camera you can contact us back again and we will surely go ahead and assist you. You can also revert back to this email for further assistance or contact us back again for further assistance.
What software do you recommend that I can use to test this card?
I understand your concern regarding the card showing errors however would like to inform you that we do not have any suggested software to test our cards and if you face any issues while using this card on your camera then please contact us back again with your incident ID1735950 or revert back to this email for further assistance and we would be glad to replace the card for you.
Although it's good to know SanDisk will replace defective cards that won't help end-users recover corrupted files on the card!
A search on Google pulls up scant few Flash Memory test diagnostics, freeware or paid. I did find one called 'CheckFlash 1.16' written by a programmer based in the Ukraine. It is fast and uses two data patterns (Hex5, HexA) to insure all memory cells can be written to both 1 and 0 and read back correctly. The Flash Drive Test Utility 1.14 and others write a single test pattern and then read back and compare the data. This is fine to check the actually memory size to find "fake" cards and USB drives, but won't insure data integrity.
There is only a Windows version, but it runs fine on my Windows 7 64bit system. The SanDisk Extreme 32GB 120MB/s UDMA7 card passed with no issues! Select your memory card in the dropdown 'Drive' selector and then use the settings as in the below screenshot. THIS WILL OVERWRITE ALL DATA, so first backup any files on the card to your hardrive.
The 'Small pattern test' is alternating 1010 data followed by 0101 data on the 2nd pass. This insures insure all memory cells on the card are functioning properly. The 'Full pattern set' is much more extensive and will take quite a long time to run on large cards. If you have a card that has issues and it passes the 'Small pattern test' then you might want to run the 'Full pattern test' it before tossing the card in the garbage. If it passes both tests you probably have other issues.
Use a good high-speed card reader, preferably USB3 or FireWire. You will need to Format the card when you replace it in the camera.
Message was edited by: trshaner Corrected 0101 data test pattern.







