Lightroom and Photoshop issue with Raw files from my D500

[allanh74012973]

Since I bought my Nikon D500 earlier this year I have had an issue where about 5% of the files ended up looking like the screenshot below. A couple of months ago the exported jpegs had the issue. When I exported this raw file the jpeg was fine. The problem is apparrent on Lightroom and Photoshop CC 2017 on Windows and on my Mac . When I open the file. In Windows 10 "photos" app the problem is not shown, but in Apple's preview it is shown. The memory card is a Lexar XQD 2933x card. The D500 saves the raw files as 14 bit raw compressed lossless.

exported jpeg:

[Just Shoot Me]

That is a corrupted file. It could be the camera, memory card, card reader you are using or even the cable connecting the card reader to the computer or cable if you connect the camera directrly to the computer that is causing the corruption. Try a different memory card first. Card reader second. New cable third or if you are connecting the camera directly to the computer try a card reader.

If none of the above fixes the issue then the camera is at fault and will need to be fixed by the manufacturer.

[Jao vdL]

Apparently there was an issue with certain Lexar cards and the Nikon D500: Official Lexar Response on D500 | DSLRBodies | Thom Hogan . Make sure you have updated your camera's firmware. Might help: Nikon | Download center | D500 Firmware

[allanh74012973]

I saw that before but it relates to SD cards not XQD.

update:

I have updated the firmware to 1.11, the images direct from the memory card look fine.

[Jao vdL]

If the images look fine in camera from the memory card, it does not mean the raw file is fine. When you look at a raw image in camera (or in Nikon View), it show you the embedded jpeg file that every raw file has. It does not show you the actual raw data. When you open in Lightroom or in apps like Apple preview, it actually reads the raw data and renders that. So when the image looks corrupted in Apple preview, Lightroom or photoshop, but not in Nikon view or on camera it means the raw data got corrupted somewhere along the chain. This means the camera itself, the card, the USB cable (if used), or the card reader has a problem. Eliminating which one it is is then the problem. Test using a different card, a different USB cable and a different card reader. If none of those help, it is probably a problem with the camera.

Lastly, make sure you NEVER use Nikon's software to transfer raw files. It is well known to corrupt Nef files if you are not using the very latest version. It turns files unreadable in anything but Nikon software.

[cmgap]

Are you confirming that the only time you see this on a Mac? Photoshop and Lightroom on the PC the issue isn't there? Do you see it at the finder level on a mac or only in Photoshop or Lightroom?

[Jao vdL]

Are you confirming that the only time you see this on a Mac?

From the first post:

The problem is apparrent on Lightroom and Photoshop CC 2017 on Windows and on my Mac

[cmgap]

Does this happen only with images imported from this card? Is this the only card you have? What happens if you make a smart preview of the image you've shown in your screen shot?

[allanh74012973]

I havent used any other XQD card, I do have another Lexard XQD card which I plan to use as a test next tim e im out shooting. The problem occurs when I try to import that known corrupted file with smart preview. As i said earlier the files from the memory card appear to be not corrupted.

[Just Shoot Me]

allanh74012973  wrote

As i said earlier the files from the memory card appear to be not corrupted.

And the question was asked how do you know, where and how are you viewing, that file is not corrupted outside of either LR or PS?

What you are viewing outside LR or PS is the Embedded JPG file, Not the actual RAW file. The Embedded JPG may be fine and that does not mean the RAW data is not corrupted.

This is not a LR or PS problem. LR and or ACR has not corrupted the file. The camera, the card, or the other hardware you are using to get the RAW file from the card to the computer is the thing that has corrupted the image.

[rodinaustin2015]

Ok.  I'm having a similar issue and here's my work flow process.  Any useful comments appreciated here:

1.  Import RAW images into my IPAD Pro using my Apple SD Card reader.

2.  Once all images are done importing, I run LR on my IPAD and images are imported into LR.

3.  In LR, rate images and maybe do some slight edits in case I want to post on social media.
4.  Go to my Win 7 PC, start LR and I may notice that SOME of the files show up as corrupt.

5. Go back to Ipad Pro LR and notice that now I see some images as showing up as corrupt.

6. Check Camera Roll, Raw Images appear to be fine.

7. Plug Ipad Pro into PC, copy images out of DCIM libraries and overwrite the few RAW Images and LR shows them correctly without any corruption.

Now, I have used multiple card readers, multiple SD Cards and the problem continues.  So, either the camera is writing bad data (doubtful as I can pull the images from the IPAD DCIM source), the Ipad is corrupting the images via the import into camera roll (again, I'm using these files to recover my data so doubtful), or there's an issue with the LR import that we aren't seeing (Doubtful by everyone's guess that's been commenting here). 
So I'm in need of a little bit of help in trying to determine where the file corruption is happening.  Telling me LR isn't doing it isn't helping narrow down the problem. 
Regards,
Rod

[cmgap]

1.  Import RAW images into my IPAD Pro using my Apple SD Card reader. Import RAW images into your Camera Roll? Have you initiated an import directly into Lr Mobile to bypass camera roll?

4.  Go to my Win 7 PC, start LR and I may notice that SOME of the files show up as corrupt. Are you getting sync errors? What happens if you view in Lr Mobile via a browser at lightroom.adobe.com?

5. Go back to Ipad Pro LR and notice that now I see some images as showing up as corrupt. Does your Lr mobile app need to be updated?

[rodinaustin2015]

1.  Import RAW images into my IPAD Pro using my Apple SD Card reader. Import RAW images into your Camera Roll? Have you initiated an import directly into Lr Mobile to bypass camera roll? 

---Interesting comment.  I'm not sure I know how to do that.  If there's a way to bypass the import into Camera Roll I'll give that a try.  Currently, I stick my SD card into the reader, Photo App pops up with the Import Screen and photos are imported into Camera roll.  This is the only way I know how to import into my Ipad.  NOTE: I don't normally lug the laptop around, but I'm going to test the next set of imports using my Win7 machine.  My preference would be the Ipad as I specifically purchased it for this reason. 

4.  Go to my Win 7 PC, start LR and I may notice that SOME of the files show up as corrupt. Are you getting sync errors? What happens if you view in Lr Mobile via a browser at lightroom.adobe.com? 
---There aren't any Sync errors.  The corruption is seen on the web and I'll confirm this once I can pull up my LR app later today.

5. Go back to Ipad Pro LR and notice that now I see some images as showing up as corrupt. Does your Lr mobile app need to be updated? 
---All apps are running at current release.  This problem has been happening now for a few months and I've noted a few app updates in the time frame.

Thanks for the on spot questions.  This is really helpful.

[Jao vdL]

---Interesting comment.  I'm not sure I know how to do that.  If there's a way to bypass the import into Camera Roll I'll give that a try.

This is not possible on iOS. You have to go through the Photos app first. This is an annoying shortcoming on iOS and is Apple's fault for not exposing the card reader to other apps.

---There aren't any Sync errors.  The corruption is seen on the web and I'll confirm this once I can pull up my LR app later today.

What is likely happening is that the raw images are already ciorrupt on your card. When the photos app pulls them in what you see is the jpeg previews embedded in the raw files. They are often not corrupted when file corruption happens on the card so you often won't notice. When you import in LR/mobile, it will show you the same jpeg preview, UNTIL you go and change some develop settings. Only then will it render from the raw data. A similar thing happens when you sync to LR web or on your computer. Only then will it render from the actual raw data.

[rodinaustin2015]

Jao,

   Perhaps you missed my statement earlier:
1.  Once LR shows the image as corrupt, I can go to my Ipad, copy the image from the DCIM folder and overwrite the catalog image.  At this point, LR no longer sees any corruption on any of the systems: Ipad/Android Phone/Win 7 system.

2.  I can now modify the image on any system and it never goes back to being corrupt...

So as far as I can see, the images made it into the LR Camera Roll without corruption since that's the source of my good files.  What can you folks suggest before my next import to narrow down the problem?    I could import the images and make a backup of them from the DCIM directory and import them into a new catalog on my Win 7 desktop.  The bits are all the same, the difference is how they get imported into a LR catalog.

What do you think?

[Jao vdL]

Ah. I did not realize that. So you copy the images over again from Photos into LR/mobile and the corruption disappears, or do you go back all the way to the card? It sounds like the images are getting corrupted in the sync up to the cloud. That would be quite strange.

[rodinaustin2015]

Jao,

  A slight correction: I copy the RAW files from the Ipad Photos location to my Win7 LR folder where the image is stored.  I don't have any way of replacing the file in LR/Mobile filesystem on the Ipad. 

Your Sync comment has me considering another possibility which at the moment seems to make a lot of sense to me...

Consider this:  LR Catalog and images were stored on an internal SSD until January when I started a photography course.  As as course requirement, I had to move the catalog and images onto a USB 3.0 drive.   There were time at dismount that I wasn't real sure that the drive had been properly shutdown on the Mac or PC.  I wonder at this point if files are making it off the Ipad, into the cloud and down to my Win 7 machine only to be corrupted by an improper shutdown of some sort.      On LR startup on my Win 7 machine, it loads a corrupt image and this is pushed up to the cloud and Mobile devices.

I haven't seen the problem this week, but classes ended this week.  So I've had a break from camera and the USB drive porting back and forth from the lab.

I'm going to move the catalog and images back onto the internal drive and see if the problem resurfaces in the next few weeks.

Any thoughts here would be greatly appreciated.

Regards,

Rodney

[Jao vdL]

That is actually not unlikely that the corruption is coming from the external disk if it hasn't been unmounted properly. They can be quite finicky this way.

[rodinaustin2015]

Hello folks,

  Well, since our last discussion I did the following:
1.  Copied the LR catalog and images to a new system so that these guys reside on an internal drive.

2.  Win7 is also a clean install.  So system was started up from scratch.

3. LR on my Win7 system is always closed until after I've finished my import of images on my IPAD Pro via the Lightning card reader.

4. All seemed well until this past weekend after shooting several hundred images in 4 different batches.   The corruption is back.

If you recall, I import images into the LR Camera roll via the reader (I've also tested with my EyeFi cards via Wifi).  Once the import is done, I start LR and it imports the images.   In the last 4 imports, I've seen immediately that a few of the RAW images show up as corrupt on the IPAD immediately.

If I take the images from the DCIM directories on the Ipad and compare them to the LR files, the corrupt ones are clearly a different size.  Slightly different, but different.

If I copy the corrupt images from the DCIM directory and copy to the LR location of the RAW images, the files will now show up correctly in LR Mobile and Desktop.

Seems to me that I'm back to thinking there's an issue with the import process or bad storage space on the IPAD.  Doesn't make sense since the original files are ALL GOOD images. 
How do I debug this further?

Regards,

Rodney

[Jao vdL]

That does look an awful lot like a bug in Lightroom's importing process. I suggest you post your findings on https://feedback.photoshop.com . Actual Adobe engineers hang out there they might look into this. Over here it is just other users mainly.