9 Replies Latest reply on Oct 30, 2011 1:48 PM by Noel Carboni

    DNG converter and USB 3.0

    redcrown on guard Level 1

      I replaced a 6 year old Sandisk USB 2.0 compact flash card reader with a new Kingston USB 3.0 reader. Was hoping to get faster transfer rates using the Adobe DNG converter to xfer and rename directly from the compact flash card to hard disk.

       

      Didn't happen. The conversion/xfer times are identical. But if I just copy files from the compact flash card readers to hard disk using Windows Explorer drag-and-drop, the USB 3.0 unit is about twice as fast.

       

      Using a random batch of 40 raw files, it takes the DNG Converter 46 seconds to complete the job using either card reader. Using Windows Explorer to just copy, the USB 3.0 unit does it in 14 seconds while the USB 2.0 unit does it in 28 seconds. So USB 3.0 is twice as fast.

       

      I didn't expect the DNG Converter to be twice as fast. Part of its job is conversion and part is I/O. I would guess 60% to 70% for conversion, leaving 30% to 40% for I/O. So I thought a 50% reduction of the I/O time would be reasonable.

       

      What am I missing in this logic?

        • 1. Re: DNG converter and USB 3.0
          Noel Carboni Level 7

          What you see makes perfect sense if your system / the DNG converter are doing I/O at the same time as processing.  Hardly anything in the modern computing environment is done serially any more.

           

          That said, I did some measurements and found that if I use Camera Raw instead of the standalone DNG converter to do the conversions, the time to convert a group of images is literally cut in half.  Camera Raw may be doing more or better multithreading.  The .dng files don't quite come out identical, but they both seem to work equally well.  I think I'll start a thread to ask about these differences...

           

          Out of curiosity, how long does it take to convert the same 40 files in a folder on your hard drive?

           

          Correllary question:  Are you not saving your original camera files?  Making DNGs is a great idea, but I'd definitely also keep the original out-of-camera files.

           

          -Noel

          • 2. Re: DNG converter and USB 3.0
            redcrown on guard Level 1

            Thanks for the reply, Noel.

             

            But strange, I never thought of using ACR to do the DNG conversion, so I tried it. Updated ACR to the curent 6.5 version first. My timing results were the same as using the DNG converter.

             

            Again, a random batch of 40 raw files read from a compact flash card on a USB 3.0 card reader. A total of about 388mb. The DNG converter does the batch in 47 seconds, and ACR does it in 48 seconds. Identical times given the degree of error in my punching a stop watch.

             

            If I copy the raw files to hard disk first and then use either ACR or the DNG converter to convert (HDD to HDD) I get the same 47-48 seconds.

             

            So I'm curious why my ACR times are the same as DNG converter while you say ACR is twice as fast on your system. For what it's worth, my system is Windows 7, 8GB memory, on an intel I5 quad core processor.

             

            The DNG files created by ACR are all about 50 to 55kb larger than the DNG files created by the DNG converter. But I noticed looking at the metadata in Bridge than the DNG files created by ACR have Camera Raw data in the EXIF, while the same files created by the DNG converter do not. I imagine that explains the few extra bytes. But loading the two files into Photoshop and doing a "difference" blend mode proves they are identical.

             

            In any case, I've learned that USB 3.0 gives no advantage over USB 2.0 in this application.

            • 3. Re: DNG converter and USB 3.0
              redcrown on guard Level 1

              Woops! I forgot that ACR runs as 32bit or 64bit depending on whether you launch from Bridge or Photoshop. I did my previous tests launching from Bridge. Ran again launching from Photoshop. There my batch of 40 raw files convert to DNG in 17 seconds. More than twice as fast. But still, reading from USB 3.0 or USB 2.0 appears to make no difference.

              • 4. Re: DNG converter and USB 3.0
                Noel Carboni Level 7

                redcrown on guard wrote:

                 


                So I'm curious why my ACR times are the same as DNG converter while you say ACR is twice as fast on your system.

                 

                In any case, I've learned that USB 3.0 gives no advantage over USB 2.0 in this application.

                 

                As I discovered after writing my post above, the speed difference is indeed between 32 and 64 bit.  I had forgotten about the performance difference as well (I have noticed it before)...  This led to my asking in the other thread whether Adobe is considering adding a 64 bit DNG converter to the product lineup.  With well-written code it's a pretty trivial thing to compile for 64 bit, and I'm sure a lot of pros who use the DNG converter would appreciate double the speed.  Creating a good unified installer that can be used for both 32 and 64 bit systems can be challenging, but it's doable.

                 

                Regarding the USB 3.0...  I suppose you could use it with a (USB 3 capable) backup drive to improve performance.  The current MyBooks do USB 3.

                 

                -Noel

                • 5. Re: DNG converter and USB 3.0
                  JimHess Adobe Community Professional & MVP

                  I don't think the problem is with Lightroom. Just because you purchased a USB 3.0 card reader, that doesn't automatically mean you will get 3.0 performance. If you plug that card reader into a USB 2.0 port on your computer you cannot expect USB 3.0 performance. The new USB 3.0 is backward compatible, but the device by itself cannot provide you with the expected improved performance. To get that you have to have a USB 3.0 port on the computer.

                  • 6. Re: DNG converter and USB 3.0
                    Noel Carboni Level 7

                    Jim, just keep in mind that redcrown on guard wrote:

                    Using Windows Explorer to just copy, the USB 3.0 unit does it in 14 seconds while the USB 2.0 unit does it in 28 seconds. So USB 3.0 is twice as fast.

                     

                    -Noel

                    • 7. Re: DNG converter and USB 3.0
                      redcrown on guard Level 1

                      Trust me, my USB 3.0 card reader is connected to a USB 3.0 port on the PC.

                      • 8. Re: DNG converter and USB 3.0
                        JimHess Adobe Community Professional & MVP

                        I read the original post a number of times. And nowhere within that post is it indicated that the computer had USB 3.0. It was indicated that a card reader was replaced, and that was all. Unfortunately I assumed Incorrectly that the computer only had USB 2.0.

                        • 9. Re: DNG converter and USB 3.0
                          Noel Carboni Level 7

                          No big deal.  We all miss details from time to time.

                           

                          I suppose, depending on the original transfer rate, a doubling in performance could have been seen even with just a better card reader and a USB 2.0 port, but no matter how it happened the doubling in transfer speed could reasonably have been expected to speed up his overall process - unless the system is overlapping the file transfer and data processing, which apparently it is.

                           

                          I think at this point what's really needed to halve the processing time is a native 64 bit DNG Converter build. 

                           

                          -Noel