What causes 'Unexpected end-of-file' error?

[RogierBos]

Hi,

I am a photographer and an avid Lightroom user. I have two camera's (Nikon D700 and D300) and two computers, a desktop PC and a MacbookPro. Lightroom is installed on both computers.

For the last three photoshoots I have had this problem where 1 in 3 to 1 in 5 images in Lightroom on my desktop PC suddenly gives a 'Unexpected end-of-file' error. First I thought it was my camera. But then I loaded the images onto my MacbookPro and they worked fine. Then I took some images with my other camera, loaded them on to my Desktop PC, and, voila, same problem.

The problem specified: the image stops rendering a preview. A dull preview appears. At the top of the screen the message appears: "An unexpected end-of-file occurred." The histogram window stays empty and the image can not be developed.

I have browsed through this forum some looking for the cause and the fix. There are some mentions of it, but they all date back to 2007.

I run LR 2.2 on Windows XP SP2 and on a MacbookPro with OS X, version 10.5.8.

Please help me understand the cause of this problem, and help me find a fix?

Thanks,

Rogier

Rotterdam, The Netherlands

www.rogierbos.com

[Guest]

I'd try deleting the Previews folder on the PC, with LR closed, then choosing the recent folders and force-build new Previews to see if that cures it.

[RogierBos]

Thanks for the reply. Not sure how to do that.

Where do I find that folder?

Will it delete all preview images for all the images I've loaded?

How do I 'force-build' new Previews?

I am not sure why you would think this would help - but at this point I am so desperate, I'll try anything!

Rogier

[Guest]

The Previews folder is in the same directory as the .lrcat.

Yes, it will delete all the Previews, but they will all be rebuilt anew. It might be instructive to note the size of the Previews folder before you delete it, and then after it has had a chance to rebuild the preview images.

[RogierBos]

Hi,

Since I have 42000 images in my LR Catalog I decided not to delete the whole Preview folder, but rather to create a new catalog. Into thhis new Catalog I loaded the images of yesterdays shoot. The problem was the same.

I am uploading two images that show the problem. In one screenshot the image comes up, but appears damaged (see on right); in the other screenshot the error message comes up; there is no histogram; and the image appears muted.

You might think 'the image has to be corrupted,' but Lightroom on the Mac has no problem with these files. I am tempted to think it is my camera after all: but I have two camera's, and it started happening to images from both camera's - and LR on the Mac has no problem with these images.

On the other side, in ACR I am also seeing corrupt files (see uploaded image ACR problem).

I am tearing my hair out. Why did this start to happen all of a sudden and what can be done to fix it?

[2bike]

Today is the first time I have also found this error. I was working in Hong Kong and using Lightroom on a PC. At the end of the assignment I transferred all the files to a bunch of USB memory sticks and then transferred the contents to my Mac when back home. The files that throw up this error are all from one particular memory stick (brand new/never used). They are also NEF files. Looking around on the net I see people also having problems with NEF files and CF cards or readers resulting in the "unexpected end-of-file" error.

The files had been fine on the PC in HK so the errors were something that had crept in during the transfer to the USB stick.

The memory stick is a corporate branded unit made by Chipsbrand HK and was a freebie so I hadn't used it before. I use Sandisk CF cards and they have never been a problem. Have you tried using different cards and seeing if one or other is more prone to giving this error? Have you formatted your cards recently?

Anyway, just now I ran Disk Utility on the suspect USB stick and sure enough it shows files with errors and orphaned clusters. The other sticks are clear of errors and consequently there are no problems in Lightroom. Phew! So I suggest you check your memory cards for errors and repair or reformat them. It shows it is probably worth reformatting storage cards/sticks before use to be sure of them. I do routinely format the CF cards whenever I empty them so start afresh with each shoot. This was a practise recommended in Nikon forums and has been good advice I think. I was in a hurry and didn't format the memory stick on this occasion. Fortunately the level of errors is maybe 1:25 so I haven't lost too much. I reformatted the stick a few minutes ago, transferred some files to it and re-tested and it is now error free.

Hope this helps in your search for a solution to your particular errors. I fear the images, at least in RAW format, are probably lost. Some people say that opening in a non-Adobe programme as a TIFF does at least save the basic image but obviously with no further RAW processing available. Haven't tested this myself yet.

[sfslim]

I am having the same problem on my Mac Pro as well as my Macbook Pro, so it's not just a PC problem.

It didn't happen with my D300 .NEF files, but is happening with my Leica D-Lux 4, seemingly randomly, with both raw and jpeg.

Any help appreciated.

Thanks

[2bike]

I noticed the problem after transferring the files from a work PC to my Mac via USB memory sticks. It is, I think, neither a Mac or PC thing, but more to do with errors introduced by defects in the storage media. I was also using a Panasonic LX3 (in many respects identical to your Leica) and none of its files were affected. I mentioned the NEF problems because others were speculating that maybe it was a problem limited to NEFs. Interesting that it is the other way round for you so that shows that it is not just a Nikon thing.

Have you tried formatting the SD cards that you use with the Leica? What make are they/it? Have you looked at the card(s) in Disk Utility to see if it shows errors (select your card when it appears in the Disk Utility window and use Verify Disk)?

[sfslim]

I always format my card in the camera whether the Nikon or the Leica. it's a 2GB San Disk.

This doesn't happen with my CF cards.

I have run Disk Utility on the SanDisk, Repair Permissions etc.

Your suggestion that there could be a problem with the reader is a good one. I used one cheap reader with the Mac Pro originally and that may damaged some of the files by the time I tried them in the pc slot reader on my MacBook Pro.

By the way I really appreciate you all getting back to me so quickly.

I will get another reader and shoot off some uninportant pictures and get back to you. The ones I lost were of our Govenor.

All of this did shut down LR 2.5  (as in I could not open it again, even after rebooting and reinstalling). I'm going to stick with LR 2.4 for a while.

[2bike]

Are you using the same card in both cameras (I ask because your post implies that it is one SD card only)? I have heard that it is a bad idea to mix cards and cameras as the formatting is different for each make. Also I believe repeated formatting eventually degrades the card as there are a limited (though quite large) number of formatting cycles that can be performed. Did Disk Utility reveal that there were errors in the files on the card? Maybe get a dedicated SD only reader rather than a multi-card one. I have one that I picked up from an HP trade stand and it has been very good. My last trip that threw up the file errors I had a cheap Integral reader. However the files were uncorrupted on the PC and in my case I'm sure the damage was caused after the initial download when they were transferred to the dodgy USB stick.

The "shut down" of Lightroom sounds serious. So far 2.5 is working fine for me.

[Ian Lyons]

I run LR 2.2 on Windows XP SP2 and on a MacbookPro with OS X, version 10.5.8.

Try upgrading to Lr2.5

[2bike]

It happened to me with version 2.4 and there is no change in 2.5. Lightroom is flagging up an external problem. Photoshop (or rather ACR) has the same trouble with the damaged files.

[Ian Lyons]

2bike wrote:

It happened to me with version 2.4 and there is no change in 2.5. Lightroom is flagging up an external problem. Photoshop (or rather ACR) has the same trouble with the damaged files.

In the past, the damage shown in the linked files has been tracked to an issue with USB port or device. I don't know if this is the case with your images, but it's worth checking. Trying a different card reader or connecting via FireWire are some of the things I would try.

[2bike]

Thanks Ian, and I am pretty certain now that in my case the problem indeed stemmed from one particular USB memory stick. Yesterday I erased its contents and after adding new files to it it appeared OK. Today I checked it in Disk Utility again and it showed problems with a number of the files, once again reporting orphaned clusters etc. I ran the Repair routine on it and it says it has repaired the volume. We'll see - I'll test again tomorrow. If it shows further problems I'll bin it as it just isn't worth the risk to further files.

It would be interesting if someone had the files that were reporting damaged in Lightroom still on their original as-shot card (I'm afraid i don't) and could try different methods of transfer to see if the errors are being introduced by a particular device as you say. If the original memory card and files are OK then the errors may not show up in all methods. However, if the problem is a corruption on the original card then, in theory, all transfer methods would reveal the same errors.

[FlyingE]

I recently had the same problem when importing about 3,500 NEF, shot with a Nikon D700 and imported into LR 5.5 on Power Mac.  Out of the 3,500 photos there were about 40 that had an end-of-file error.  Fortunately, I had the original photos still on the card, and using the same reader but a different USB port, I was able to import the previously defective files without any errors.

[rxfelix]

While the corruption in your .NEF file could come from any one of a number of sources, here is a way to recover at least some of the information if you own Adobe Acrobat.  Your .NEF file should contain a smaller JPEG file created by your Nikon camera when the picture was taken.  Even if the .NEF file is corrupt, the JPEG may well be intact, and you can tell that's the case if a preview is visible in Lightroom or Bridge. 

If you try to print it from Bridge or Lightroom, though, you will get an error.  You can get around this by loading up a CODEC that will let you preview the files in Windows Photo Gallery.  Nikon has one available for download to use with Vista x32 at http://www.nikonimglib.com/nefcodec/index.html.en, but it doesn't support Vista x64, so if you're running that (as I am) you'll need a second party CODEC such as the one available from http://www.ardfry.com/Products.htm with a 15-day trial download available.  They also have a DNG CODEC which I have not tried.

With that CODEC installed and your system rebooted, preview the damaged NEF file in Windows Photo Gallery.  Choose the "Print" option and select the "Adobe PDF" printer.  Choose the "Postscript Custom Paper Size" option, and set the dpi resolution to the highest available (4000).

Go to the "Options" link on the bottom right of the window and uncheck the "Sharpen For Printing" box, as you can do this yourself with later processing.  Then go to the "Printer Properties" link to see the Adobe PDF settings.

Go to the Adobe PDF Settings tab and choose Default Settings: High Quality Print, which will set the resolution to 2400 dpi. 

You want to have Acrobat interfere as little as possible with the output, so select "Edit" from the Default Settings tab, which will bring up the "High Quality Print - Adobe PDF Settings" dialog.  Here, turn off the "Optimize for fast web view" check box, then go to the "Images" folder and set the "Downsample" setting to Off and the pixels per inch to 2400.  Also, set "Compression" to Off.

If you're going to do this in the future, you can also "Save As ..." your settings for future use with a name like "Fix Bad NEF Files."

Print the file to a designated folder.  It won't be quite as good as the NEF file would have been, but at least you haven't lost the shot.

[IanHylands]

I have had the same problem with LR 2.5 and ACR 5.5 now for several months. I have the problem with NEF files from my Nikon D3 and D700. I was using Lexar Pro UDMA cards and a Lexar firewire 800 reader on my Macbook Pro when it started happening. I have since switched to brand new SanDisk Extreme Pro cards and a SanDisk Firewire 800 card reader, with no change in the problem. It is intermittent. Some cards will will import fine one day, and not the next. The problem seems to be Lightroom Centric, if I import files using Lightroom and they are corrupt I can usually drag and drop the files from the card into a folder and then import them and they are fine. Does anyone actually have a solution to this problem or should I try to find a different software import option?

[2bike]

Just a thought. Have you, or anyone else on the thread, tried doing a complete reinstall of Lightroom? I have had no problems with importing on my Mac since encountering the issue and wonder if it was the other PC installation that triggered my problems - not because it was a PC, this issue seems to be one that affects both platforms, but maybe there was some glitch in that particular installation. Another thought, which may be a complete red herring - that PC installation was based on the downloaded trial software and my untroubled Mac install is from the purchased disc.

[IanHylands]

I've switched to a brand new Mac Book pro with the same problems, and my install is from the DVD, not the download. It didn't always happen, but it seems to happen more and more regularly. It's usually not a problem if I transfer the files to the correct folder by another method and then import from LR without moving them. So it seems to be an issue LR has when moving files from one location to another. I just can't figure out why it would be getting worse over time...

[2bike]

Have you tried the LR3 beta? What happens if you make duplicates of a folder that has shown problems and then run them into a new LR3 catalogue as a test (given that the beta won't work with existing catalogues)? I know when i got problems with the USB stick i wasn't using Lightroom to do the transfer, just copied the folders direct to the stick. I only noticed the problem when reopening the catalogue and files on my home machine.

[IanHylands]

I don't really have time to mess around like that. Now I just transfer everything manually from the card to a folder and then import into LR from there. That seems to prevent any problems. I'll be switching to the 2.6 Beta this week when I pick up my D3s so I'll let you know how that works.

[mwgarrett]

I also ran into this problem 'Unexpected end-of-file' error. My Photoshop document took several hours to complete and then it was unable to open. My fix was to go back in time (using time machine, in OS 10.6.2). I selected the file from an earlier backup and voila. It would be nice to find out what is causing the problem in the first place though.

mg

[IanHylands]

I still haven't heard anything from Adobe on this, and same with a few other people. It seems like they're too busy to deal with this kind of stuff if they don't have a ready answer. Too bad. I haven't had any problems yet with 2.6, but I'm also not using an Adobe product to transfer the files from the card which is where my previous problems seem to have come from.

[Winjeel]

I've had this happen for the first time just now. I've imported a huge number of images taken yesterday at a sumo tournament, using a

brand new memory card. I've gone through flagged the ones I want in the Library module, then as I'm going through them in the Develop module, I'm using a preset I made (a white balance adjustment) and one of the flagged files gave me that message. Since others believe they've traced the source of the error to their memory sticks... I'm not sure if the error occurred because of the new memory card, or from the external hard drive I'm using. I'll admit, the external hard drive is about 4 years old, it's summer, it doesnt have any ventilation or cooling of its own, but its a small 20Gb pocket drive. Now I'm getting worried.

Edit: just adding, I'm using LR v2.7

[Jonathon_LeFaive]

Did you format the new card in your camera before you shot with it?