29 Replies Latest reply on Jul 31, 2015 6:21 AM by trshaner

    converting some canon raw (cr2) files to dng gets corrupted

    merkk Level 1

      I imported a batch of photos off my camera's media card and for some reason, 3 of the photos consistently get corrupted during the dng conversion process. lightroom gives no sort of error message.

       

      Any suggestions on how to work around this other then just not converting them to dng? I'm attaching a jpg version of what one of the photos looks like after being converted dng

       

      edit: correction - this corruption happens even if i just do a straight copy and don't convert it to dng. I did manage to import and convert 2 of the 3 files. I didn't change anything - they just worked after trying it again. But the 3rd file i've tried multiple times and it always gets corrupted. I even tried copying the file manually to my C drive and then importing from that, still gets corrupted. I did finally get it to work by manually copying the photo in the folder i would have had lightroom copy the file to, doing any import/add (no copy no convert) and after importing, then converting it. So it seems the issue was actually with adobe actually copying the file. Seems odd since that should be the most basic thing it's doing.

       

      IMG_2277.jpg

        • 1. Re: converting some canon raw (cr2) files to dng gets corrupted
          Just Shoot Me Adobe Community Professional

          Then the actual files are corrupt. Might be time to get a new memory card.

          • 2. Re: converting some canon raw (cr2) files to dng gets corrupted
            merkk Level 1

            If the actual files are corrupt, then manually copying the files and just having lightroom do an add (no copy or move) wouldn't have made a difference. Plus the files are perfectly fine when i view them outside of lightroom. it is only after lightroom moves or copies the file that they become corrupted.

            • 3. Re: converting some canon raw (cr2) files to dng gets corrupted
              99jon Adobe Community Professional & MVP

              Try the stand alone converter. Put your CR2 files in a source folder and choose destination folder. Then import the DNG files from the destination folder. You can get the latest DNG converter 9.1 for Windows or Mac from the following link. The DNG files should then work with your present software (keep your originals as backups and for use in the camera manufactures software)

               

              To download click here - DNG Converter 9.1

               

              You can convert a whole folder of raw images in one click. See this quick video tutorial:

               

              You Tube click here for DNG Converter tutorial

              • 4. Re: converting some canon raw (cr2) files to dng gets corrupted
                merkk Level 1

                Please see the edit I made to my original post. It turns out the problem wasn't in converting the files it was in adobe just moving or copying the original file even though the original files appears to be completely fine. See the edit I made to the original post

                • 5. Re: converting some canon raw (cr2) files to dng gets corrupted
                  Just Shoot Me Adobe Community Professional

                  If you are looking at the files with something like Windows photo viewer or Mac Preview all that is showing is the embedded JPG preview image which might be fine.

                   

                  Open those files in  Photoshop through ACR and see if they look OK.

                  • 6. Re: converting some canon raw (cr2) files to dng gets corrupted
                    dj_paige Level 9

                    As the others have explained, you have a hardware problem. It could be camera card, transferrring to computer, hard disk or memory.

                     

                    The fact that you think the photo is "completely fine" is because most viewers show you the embedded JPG, and not the full RAW. The embedded JPG is "completely fine" and the RAW is corrupted. That's what has happened. The RAW was corrupted somewhere before it got to Lightroom.

                    • 7. Re: converting some canon raw (cr2) files to dng gets corrupted
                      merkk Level 1

                      dj_paige wrote:

                       

                      As the others have explained, you have a hardware problem. It could be camera card, transferrring to computer, hard disk or memory.

                       

                      The fact that you think the photo is "completely fine" is because most viewers show you the embedded JPG, and not the full RAW. The embedded JPG is "completely fine" and the RAW is corrupted. That's what has happened. The RAW was corrupted somewhere before it got to Lightroom.

                      Ah, ok i understand your point now. Although i still think your point is incorrect about it being hardware related. If it were hardware related, and the embedded jpg is ok while the raw is corrupted, then my copying the file manually using windows explorer and having lightroom just add the photo to the catalog instead of copying or moving the file should have also resulted in a corrupted image, no?

                       

                      I also just tried opening the files in PS ACR as shootistbond007 suggested and the files opened ok - i opened them directly off the media card.

                       

                      So i really don't think this is a hardware issue on my end or a corrupted file. It seems like a glitch in LR.

                      • 8. Re: converting some canon raw (cr2) files to dng gets corrupted
                        dj_paige Level 9

                        It's a hardware malfunction. It's always a hardware malfunction. And it can be intermittent. If you do it just now and its fine, and you did it a previous time and its corrupt, then ... ???

                         

                        It's not Lightroom. There's no plausible mechanism for Lightroom to cause this problem. Lightroom uses operating system calls to copy and read the file. There are plausible mechanisms for hardware malfunctions (which can be intermittent) to cause this problem.

                        • 9. Re: converting some canon raw (cr2) files to dng gets corrupted
                          merkk Level 1

                          well, since i can't see the code for LR or debug it i can't really agree or disagree with you. From my experience, it seems like it was a bug in lightroom. But i realize that's a limited test sample so you could be right as well.

                          • 10. Re: converting some canon raw (cr2) files to dng gets corrupted
                            Just Shoot Me Adobe Community Professional

                            merkk wrote:

                             

                            well, since i can't see the code for LR or debug it i can't really agree or disagree with you. From my experience, it seems like it was a bug in lightroom. But i realize that's a limited test sample so you could be right as well.

                            You are the second person in 2 days that has this problem. The other poster is using a Canon 6D. What model Canon camera are you using.

                             

                            Now if it was really a LR problem there would be a multitude of post on this and many other forums about it and there simply isn't.

                            • 11. Re: converting some canon raw (cr2) files to dng gets corrupted
                              merkk Level 1

                              just because there aren't a lot of posts, doesn't automatically mean it's not a problem with LR. If it is a bug in LR it could be one that only happens under very narrow circumstances. For example (and i should have mentioned this in the original post, sorry) I'm using windows 10. I'm guessing there's only a small percentage of people using LR under windows 10 at this point. Maybe this only happens under windows 10.

                               

                              To answer your question, I'm using a canon rebel T3I

                               

                              Question for you - when LR moves or copies a file, but does not convert it, does it also create a new jpg preview? One thing i noticed is that during the import process, the photo would show up in the gridview and for the first second or two that it showed up it looked ok, and then it would change over to the corrupted version. This happened just doing a move or copy, no conversion.

                              • 12. Re: converting some canon raw (cr2) files to dng gets corrupted
                                Just Shoot Me Adobe Community Professional

                                Answers to your questions

                                No unless you select to convert it to a DNG.

                                No it does not create a JPG. To my knowledge all RAW files contain a preview JPG image. That is what is displayed on the camera LCD screen, not a RAW file.

                                 

                                LR does create a Preview File from the RAW data. But that only happens when you are viewing the image at a 1:1 scale. That is unless you select to create a 1:1 preview on import. Then it creates that file when imported.

                                • 13. Re: converting some canon raw (cr2) files to dng gets corrupted
                                  dj_paige Level 9

                                  Question for you - when LR moves or copies a file, but does not convert it, does it also create a new jpg preview? One thing i noticed is that during the import process, the photo would show up in the gridview and for the first second or two that it showed up it looked ok, and then it would change over to the corrupted version. This happened just doing a move or copy, no conversion.

                                  Again, this is not indicative of a Lightroom problem.

                                   

                                  Whenever a RAW is imported into Lightroom, it first shows the JPG preview embedded in the file for a few seconds until it can generate its own preview from the RAW. In the case where the RAW is corrupted before it gets to Lightroom, then in this case, it changes from the JPG preview to the corrupted RAW photo. The process of generating its own preview does not corrupt the file, it simply shows what is in the file. And at no time during this import process is Lightroom writing to the file itself, so there is no plausible mechanism for Lightroom to cause this corruption.

                                  • 14. Re: converting some canon raw (cr2) files to dng gets corrupted
                                    trshaner Adobe Community Professional & MVP

                                    If ACR opens the CR2 file with no issue directly off the memory card and LR opens the files corrupted this indicates an issue with the 'destination' hard drive. Try copying the files to a folder on the same hard drive using Explorer and then try to edit them using ACR and LR? If the same issue is present in both ACR and LR then you have a failing hard drive or related hardware issue.

                                    • 15. Re: converting some canon raw (cr2) files to dng gets corrupted
                                      merkk Level 1

                                      trshaner wrote:

                                       

                                      If ACR opens the CR2 file with no issue directly off the memory card and LR opens the files corrupted this indicates an issue with the 'destination' hard drive. Try copying the files to a folder on the same hard drive using Explorer and then try to edit them using ACR and LR? If the same issue is present in both ACR and LR then you have a failing hard drive or related hardware issue.

                                      I already did that, and it worked. That was the only way i could get LR to import the files. I copied the files myself from the sd card to the folder I would have had LR copy the files to. Then I opened lightroom and told it to just add the files to the catalog - and that worked.

                                      • 16. Re: converting some canon raw (cr2) files to dng gets corrupted
                                        merkk Level 1

                                        dj_paige wrote:

                                         

                                        Question for you - when LR moves or copies a file, but does not convert it, does it also create a new jpg preview? One thing i noticed is that during the import process, the photo would show up in the gridview and for the first second or two that it showed up it looked ok, and then it would change over to the corrupted version. This happened just doing a move or copy, no conversion.

                                        Again, this is not indicative of a Lightroom problem.

                                         

                                        Whenever a RAW is imported into Lightroom, it first shows the JPG preview embedded in the file for a few seconds until it can generate its own preview from the RAW. In the case where the RAW is corrupted before it gets to Lightroom, then in this case, it changes from the JPG preview to the corrupted RAW photo. The process of generating its own preview does not corrupt the file, it simply shows what is in the file. And at no time during this import process is Lightroom writing to the file itself, so there is no plausible mechanism for Lightroom to cause this corruption.

                                        You and shootisbond007 said the opposite - so not sure which of you is correct. Although given the behavior of LR your explanation seems more likely.

                                        • 17. Re: converting some canon raw (cr2) files to dng gets corrupted
                                          trshaner Adobe Community Professional & MVP

                                          merkk wrote:

                                           

                                          trshaner wrote:

                                           

                                          If ACR opens the CR2 file with no issue directly off the memory card and LR opens the files corrupted this indicates an issue with the 'destination' hard drive. Try copying the files to a folder on the same hard drive using Explorer and then try to edit them using ACR and LR? If the same issue is present in both ACR and LR then you have a failing hard drive or related hardware issue.

                                          I already did that, and it worked. That was the only way i could get LR to import the files. I copied the files myself from the sd card to the folder I would have had LR copy the files to. Then I opened lightroom and told it to just add the files to the catalog - and that worked.

                                          Feel free to try the below possible solutions in whatever order you prefer.

                                           

                                          1) This indicates the hard drive is probably OK, but you may have a system memory issue. You can download the free edition of MemTest86 here:

                                           

                                          MemTest86 - Offical Site of the x86 Memory Testing Tool

                                           

                                          2) Another possibility is that your card reader or memory card is failing when LR transfers the image files due to higher data transfer speed. You can test this by trying a different card reader or with the camera connected directly via USB. If still corrupting the image data try another "known good" memory card.

                                           

                                          3) There is a remote possibility that your LR installation is causing the image file corruption. To determine if this is the cause a) uninstall LR, b) delete the LR Preferences as outlined at the below link, and c) reinstall the latest LR update for the version you are using (3, 4, 5, 6, CC).

                                           

                                          http://www.lightroomforums.net/showthread.php?25197-Resetting-the-lightroom-preferences-fi le-updated

                                          • 18. Re: converting some canon raw (cr2) files to dng gets corrupted
                                            merkk Level 1

                                            LR transfers the images faster then just copying them off the card using explorer?

                                             

                                            Since LR is the only app on my pc right now having any sort of issues, i still tend to think it's not a hardware issue. I appreciate the advice from everyone. If this issue becomes more problematic, I'll try one of the above steps.

                                             

                                            Thanks

                                            • 19. Re: converting some canon raw (cr2) files to dng gets corrupted
                                              trshaner Adobe Community Professional & MVP

                                              merkk wrote:

                                               

                                              LR transfers the images faster then just copying them off the card using explorer?

                                              LR uses it's own software routine to copy files. It may not be faster, but it's clearly different than Windows Explorer copy. LR is also known to stress hardware components (CPU, GPU, Memory, Storage Devices) more than most programs. There are many reports in this forum of users with systems that over-heat or corrupt files when using LR, but the vast majority of LR users have none of these issues. From the posted pictures it's clear you have hardware that doesn't play well with LR.....so what's at fault and who should fix it?

                                              • 20. Re: converting some canon raw (cr2) files to dng gets corrupted
                                                merkk Level 1

                                                I don't mean to be argumentative, but how is it clear that my hardware is the issue? I'm not 100% certain this is a bug within LR, but i don't see how anyone else can be 100% certain it's a hardware issue. Just because most people don't report having this problem does not automatically mean it must be a problem with my hardware.

                                                 

                                                As i mentioned, I'm running windows 10. I doubt it most people are using windows 10 at this point yet. So that's one very plausible reason why others might not be reporting this issue. And any product can have a bug that just so happens to only affect a small percentage of users.

                                                 

                                                And we might be nit picking threads at this point, but if LR stresses hardware to the point that it fails, I don't think you can entirely blame the hardware. That's sort of like me building an airplane that needs a 10 mile long runway and then blaming the airports for my plane not taking off I appreciate that some of the things lightroom does are pretty intensive. But it's not exactly an elegant design if it just rams headlong into a wall (hardware limits) rather than handling those limitations in a more graceful manner, especially if we're talking about corrupting data.

                                                 

                                                I also just checked the LR6 minimum system specs, and I'm well above the minimum specs.

                                                 

                                                Also, not sure which of you guys are correct but there's some conflicting info being posted. trshaner said LR uses it's own software to copy files. But dj_paige said "Lightroom uses operating system calls to copy and read the file". If LR does indeed use it's own code to copy files, then it would seem a little more likely that there might be a problem with LR since this problem went away when i used windows to move/copy the files rather than letting LR copy the files.

                                                • 21. Re: converting some canon raw (cr2) files to dng gets corrupted
                                                  ssprengel Adobe Community Professional & MVP

                                                  Ok, how about upload one of your DNGs to www.dropbox.com and post a public download link, here.  That way we can see if the data is corrupt on our non-Win 10 systems and in PS/ACR not LR.  Or better yet upload your raw file before you converted to the DNG as well as the DNG.

                                                  • 22. Re: converting some canon raw (cr2) files to dng gets corrupted
                                                    merkk Level 1

                                                    Dropbox - IMG_2277.CR2

                                                    Dropbox - IMG_2277.dng

                                                     

                                                    That's a link to the CR2 version and DNG of the file i posted in my original post. The current DNG version of that file appears to be ok right now, but I've uploaded a copy of that as well. And just to avoid confusion - the way I managed to get a clean DNG version of that file was to copy it using windows explorer to my destination folder, and then have LR add the file to catalog and convert it, without LR moving the cr2 file. The file also opens fine in PS. Oh - and the cr2 file i uploaded it directly off the memory card, which is how i was initially trying to import the file into LR.

                                                    • 23. Re: converting some canon raw (cr2) files to dng gets corrupted
                                                      ssprengel Adobe Community Professional & MVP

                                                      If the copy always works when Explorer is used and the copy sometimes or never works when LR is used, then I would guess LR is using different file-system or USB-driver calls than Explorer and the drivers aren't quite working correctly on Windows 10 or some other USB device isn't quite compatible and interfering, or that LR being in memory is affecting how the drivers are working.

                                                      • 24. Re: converting some canon raw (cr2) files to dng gets corrupted
                                                        merkk Level 1

                                                        well that certainly sounds like a reasonable possibility to me. IMHO Sounds more reasonable than faulty hardware since so far only LR has had an issues with corrupted files. If it happens again let me know if you want me to do anything to try and narrow down the cause.

                                                        • 25. Re: converting some canon raw (cr2) files to dng gets corrupted
                                                          trshaner Adobe Community Professional & MVP

                                                          merkk wrote:

                                                           

                                                          I don't mean to be argumentative, but how is it clear that my hardware is the issue? I'm not 100% certain this is a bug within LR, but i don't see how anyone else can be 100% certain it's a hardware issue. Just because most people don't report having this problem does not automatically mean it must be a problem with my hardware.

                                                          I didn't say it was a specifically a hardware issue: "From the posted pictures it's clear you have hardware that doesn't play well with LR.....so what's at fault and who should fix it?"

                                                           

                                                          For example, have you checked to see if there if there is a Windows 10 driver update for your card reader? Have you tried importing the same CR2 files on the same memory card into LR with it installed in your camera using its USB port? If that works then it pretty much indicates a driver or component issue with the card reader.

                                                           

                                                          merkk wrote:

                                                          And we might be nit picking threads at this point, but if LR stresses hardware to the point that it fails, I don't think you can entirely blame the hardware. That's sort of like me building an airplane that needs a 10 mile long runway and then blaming the airports for my plane not taking off .

                                                          If a system is properly designed and maintained it should run without any issues right up to 100% utilization. Insufficient cooling, dust clogged ventilation ports, low power supply voltages, or simply components that don't operate to specifications will cause issues when system resources are "heavily utilized." LR heavily utilizes system resources, which "stresses" your system. Adobe has been very careful about CPU utilization, which rarely goes over 50%. Many people have actually complained and would like to see upwards to 100% utilization. So no short runways when using LR....at least not yet!

                                                           

                                                          merkk wrote:

                                                           

                                                          Also, not sure which of you guys are correct but there's some conflicting info being posted. trshaner said LR uses it's own software to copy files. But dj_paige said "Lightroom uses operating system calls to copy and read the file".

                                                          Like most software applications LR uses both OS calls and its own routines. There's always the possibility of HW/SW compatibility issues when moving to a new OS (Windows 10). If the exact same system configuration works fine using a previous OS version then you have your answer.

                                                          • 26. Re: converting some canon raw (cr2) files to dng gets corrupted
                                                            merkk Level 1

                                                            can you tell me how you can tell from the posted picture that i have hardware that doesn't play well with LR? Is there some pattern to the corruption that indicates its a hardware issue and not a problem within LR?

                                                             

                                                            If there is a driver issue with the card reader, why does the problem only occur when LR copies the file off the card reader but not when windows copies the file off the card reader?

                                                             

                                                            I don't think there will be an updates for my card reader since there were no drivers for it at all. A driver update for the USB controller on my motherboard though... Unfortunately, there is no usb driver update available. From my experience it seems like mobo manufacturers are not very good about supplying new drivers for a new OS unless you have a very current mobo, so i'm probably out of luck in that dept.

                                                            • 27. Re: converting some canon raw (cr2) files to dng gets corrupted
                                                              trshaner Adobe Community Professional & MVP

                                                              You can post this issue as a reply at the below link and perhaps Adobe Staff Jeff Tranberry can help:

                                                               

                                                              Photoshop and Lightroom compatibility with Windows 10

                                                              • 28. Re: converting some canon raw (cr2) files to dng gets corrupted
                                                                tomasg64117537

                                                                I have the same exact problem

                                                                 

                                                                I run a Canon 7D Mark II

                                                                 

                                                                and I'm using a Mac. (brand new, bought in march) so i don't think its a hardware problem

                                                                • 29. Re: converting some canon raw (cr2) files to dng gets corrupted
                                                                  trshaner Adobe Community Professional & MVP

                                                                  tomasg64117537 wrote:

                                                                   

                                                                  I have the same exact problem

                                                                  The OP is on Windows 10 so not exactly the same. This is what is happening:

                                                                   

                                                                  1) Raw files are corrupted when imported into LR from the memory card using a card reader.

                                                                   

                                                                  2) Raw files can be copied to hard drive using the card reader in Windows Explorer (i.e. Mac Finder) with no corruption.

                                                                   

                                                                  Are you seeing the same as above?