14 Replies Latest reply: Jun 21, 2012 6:41 PM by wizard155 RSS

    Don't understand zooming a JPEG in Library

    Bob_Peters Community Member

      I am trying to understand something that has been bothering me for a long time.  It is particularly annoying now that I am eadling with Nikon D800 NEF files.

       

      OS X 10.7.4

      Mac Pro 1,1 (5 years old!)

      2X, 2.66 GHz XEON processors

      9 GB RAM

      128 GD SSD with catalog and application

      1 TB Seagate containg all data.

       

      The first thing I did was generate all 1:1 previews.  For the trace shown below I concentrated on a single preview in the Library module and performed multiple sequences of zoom-in and zoom-out.  The time samples are taken every 1/2 second (I have terrible diagnostic tools!).  The 4 cores are plotted top to bottom.

       

      The first time I zoomed to 1:1 only core number 1 was involved.  This took about 6 seconds!  Going back to the "normal" view took very little time.

       

      The second time I zoomed to 1:1 core #1 started the process and was followed by core #2 which finished the job.

       

      The third cycle looks like the second:  Cores 1 and 2 are both involved.

       

      The third cycled involved core 1 and then core 3.

       

      Patterns such as these persist throughout this and other traces I have collected.

       

      So my question:  Why is there never more than one core active at a given time when zooming to a 1:1 view?  Other applications I have seen make use of all cores.

       

       

      Screen Shot 2012-06-20 at 4.06.18 PM.png

        • 1. Re: Don't understand zooming a JPEG in Library
          web-weaver Community Member

          On my system - Win 7 64-bit - all 4 cores are used when a preview of 1:1 is generated.

          So, if it doesn't on you system, it's probably not Lr's doing.

          Keep in mind that Lr does not discard the 1:1 preview for some time. So when I go back to "normal" view and then (for the same image) again to 1:1 preview, the CPU doesn't show much activity. Stands to reason because the preview is already generated, and just zooming into a generated preview doesn'r require much CPU.

          • 2. Re: Don't understand zooming a JPEG in Library
            Bob_Peters Community Member

            web-weaver wrote:

             

            On my system - Win 7 64-bit - all 4 cores are used when a preview of 1:1 is generated.

            So, if it doesn't on you system, it's probably not Lr's doing.

             

             

            All 4 cores on my system are being used to generate the 1:1 previews.  They just take too long to do the job:  12 secomds!

             

            web-weaver wrote:

             

            Keep in mind that Lr does not discard the 1:1 preview for some time. So when I go back to "normal" view and then (for the same image) again to 1:1 preview, the CPU doesn't show much activity. Stands to reason because the preview is already generated, and just zooming into a generated preview doesn'r require much CPU.

             

            I agree that what I see doesn't make sense.  That is precisely why I'm looking for an explanation of what is going on.

            • 3. Re: Don't understand zooming a JPEG in Library
              web-weaver Community Member

              If your CPU takes 12 seonds, that means in my opinion that there is a bottleneck somewhere in your system.

              The bottleneck could be

              a) one hardware component that is not "up to speed" so that the CPU sits idle - or uses only one core;

              b) a sub-optimal configuration of your hard drives. In this case your hard drives are busy reading and writing while the CPU is on stand-by.

               

              See here what Adobe has to say about optimazition of Lr: http://helpx.adobe.com/lightroom/kb/optimize-performance-lightroom.html

               

              Regarding configuration of hard drive(s): See my post in the following thread: http://forums.adobe.com/message/4496194#4496194
              It's post # 9 - but read also # 10.

              • 4. Re: Don't understand zooming a JPEG in Library
                Bob_Peters Community Member

                web-weaver wrote:

                 

                If your CPU takes 12 seonds, that means in my opinion that there is a bottleneck somewhere in your system.

                The bottleneck could be

                a) one hardware component that is not "up to speed" so that the CPU sits idle - or uses only one core;

                b) a sub-optimal configuration of your hard drives. In this case your hard drives are busy reading and writing while the CPU is on stand-by.

                 

                See here what Adobe has to say about optimazition of Lr: http://helpx.adobe.com/lightroom/kb/optimize-performance-lightroom.htm l

                 

                Regarding configuration of hard drive(s): See my post in the following thread: http://forums.adobe.com/message/4496194#4496194
                It's post # 9 - but read also # 10.

                 

                Sorry it took so long to respond.

                 

                A request:  Please download a few D800 NEF files from the web, run them through your hardware and report the results.

                 

                I repeated the observations mentioned in my initial post on the following machine.  (What surprised me was the fact that the elapsed times were essentially the same.  Apple needs to give the MacPro a huge upgrade!)

                 

                OS X 10.7.4

                MacBook Air 4,1 (about 1 year old)

                1.7 GHz Core I5  (2 real cores and 2 virtual)

                4 GB RAM

                128 GD SSD with catalog and application

                 

                Zooming to 1:1 exhibits the same behavior:  a single real core does virtually all the work; no hyper-threading of any great significance.  The strange thing is that the work is always done by core #2.  Generating 1:1 previews involves all cores, real and virtual.

                 

                I have read the article/thread you mentioned in b) and have done the things which were possible.  Unfortunately, the MacBook Air has a single SSD so a;; I could do was alter the size of the Camera Raw Cache.  (For the Mac Pro I tried many combinations of Lightroom/catalog/data on the SSD and 2 internal 1TB Seagate grives.  I didn't see an improvement.)

                 

                Regarding your comments in a), above:  The bottleneck, whatever it is, is the "same" on 2 computers of radically different architecture.  The only obvious element which is in common is Lightroom 4.1.

                • 5. Re: Don't understand zooming a JPEG in Library
                  web-weaver Community Member

                  Bob_Peters wrote:

                   

                   

                  A request:  Please download a few D800 NEF files from the web, run them through your hardware and report the results.

                   

                  Sorry, I won't do that for various reasons:

                  a) from where on the web?

                  b) I probably would violate someone's copyright;

                  c) don't have the time to do that;

                  d) the results would be meaningless.

                   

                  Please understand that this is a user-to-user forum and as volunteers we have only so much time to dedicate.

                  • 6. Re: Don't understand zooming a JPEG in Library
                    Bob_Peters Community Member

                    a)  dpreview

                    b)  no problem with that

                    c)  oh

                    d)  No, the results might allow you to appreciate the problem.  Who knows, your D800 NEF rendering might only take a second or two.

                     

                    ...But you do have ample time to respond to posts

                    • 7. Re: Don't understand zooming a JPEG in Library
                      web-weaver Community Member

                      Bob,

                      a) What's dpreview?

                      b) But I have. I am not downloading images from somebody else's website, period.

                      d)  We two have completely different systems.

                      • 8. Re: Don't understand zooming a JPEG in Library
                        Jim Wilde Community Member

                        Bob_Peters wrote:

                         

                        A request:  Please download a few D800 NEF files from the web, run them through your hardware and report the results.

                         

                        What results do you want, exactly?

                         

                        Zooming in and out to see how many cores are being used is, to me at least, a pretty pointless exercise....zooming in and out on a few downloaded D800 nefs is instant, same as it is for all my own Canon 5DII files, so it really doesn't matter whether one core or eight cores is being used as I can't see how it could be quicker. Having said that if I rapidly zoom in and out it looks as though maybe 4 of 8 cores show some activity, though none of them are highly active. So if it truly takes 6 seconds to zoom in/out on your system(s), there's maybe something not right about your setup.

                         

                        When loading one of the nefs in Develop it takes approximately 4.5 seconds for the 'loading' indicator to disappear, which is a second or two longer than the 5DII files....but sliders are still activated sub second, which is all I'm interested in as I routinely have the 'loading' indicator turned off.

                         

                        Imported them into a new catalog with minimal previews, then rendered 1:1 previews....which took about 6.5 seconds per image. My 5DII files take on average 4 seconds to render 1:1 previews.

                         

                        Hope that helps.

                        • 9. Re: Don't understand zooming a JPEG in Library
                          wizard155 Community Member

                          You are certainly asking a lot of questions and primarily how LR and OSX interact with each other with SMP enabled software.

                           

                          One possible answer is the conversion process to a 1:1 preview might be proprietory software addon which is included in LR which is currently not SMP enabled. This type of software addons occur regularly within software houses where they purchase other software to do a job which otherwise might be cost prohibitive because of the labour intensive task of replicating the same process (i.e. re-inventing the wheel).

                           

                          OSX itself might be prohibiting LR from running more than one core for that process. i.e. How LR is doing the process might be only limited to one core. Future optimisation will fix this.

                           

                          The list of potential reasons could go on if I thought about it.

                           

                          A few things to be aware.

                           

                          D800 NEFs are nearly 75Mb in size. These are massive files comparably to lower end DSLR cameras with much less megapixels so you might have to forgive some systems being unable to process it quick enough. There is a lot of data and a lot of interpretation going on in the background. Not to mention taking into account anti-virus software and other software interfering in the process.

                           

                          I'm not entirely sure whether a 1:1 preview are actually jpegs. It might possibly be in a format which is a lossless format of converted data to represent the image rather than having to interpret the NEF or DNG (if you converted your NEFs to DNGs). Only in the minimal previews does Adobe reference that they are jpegs.

                           

                          Sometimes using a SSD isn't enough, you need to have a SATA3 compatible SSD with a SATA3 port to improve data throughput and consistently.

                           

                          Also the type of SSD can also be an issue. I'm aware that some SSD drives are just not going to cut it at all. I'm a big fan of Intel SSD and have found them extremely reliable and consistent performers. Scouting forums to verify that your brand and model are found to be reliable and consistent is recommended. Not all SSD drives are alike and the technology is only in its baby stages comparably to the rest of the HDD industry. So buyer beware.

                           

                          Most importantly is to provide as much feedback as possible to yet another baby product, Lightroom, comparably to its big brother, Photoshop. Software development can be difficult and to build a product from the ground up needs feedback and whether you can reproduce the same issue(s) reliably. This will help bugs to be ironed out and resolved in the hope of turning out a better, faster, stronger product.

                           

                          Even though it doesn't look like it, Adobe developers do pay attention to you airing your grievances and is also the reason why they supply release candidates to the software to the general public. The more testing by as many people result in a greater feedback pool and no matter the amount of testing, someone will always find an issue.

                          • 10. Re: Don't understand zooming a JPEG in Library
                            Bob_Peters Community Member

                            Thank you!

                             

                            That is exactly the information I am looking for.  Zooming D800 preview files to 1:1 on both of my systems is painfully slow.

                             

                            Your system comutes the 1:1 previews in about the same amout of time as the Mac Pro I just returned to Apple.  But even that systen was much slower than "instant" when zooming to 1:1.

                             

                             

                            Thanks again for the information.

                            • 11. Re: Don't understand zooming a JPEG in Library
                              trshaner MVP

                              Windows 7 64 bit, 2.8GHz i7-860, 12 GB, LR4.1 final

                               

                              All eight threads (hyperthreading enabled quad core processor) are used to build and update the previews on this system.

                               

                              11 sec. to build the initial D800 NEF preview in the Library module

                              0.25 sec. to Zoom 1:1 and build image

                              2 sec. to 12 sec. to update 1:1 Library preview after making changes in the Develop module, depending on what is changed.

                               

                              Overall LR4.1 final seems about 1.5x to 2x times slower with the D800 NEF than my 5D MK II CR2 files (20Mp), but still acceptable.

                              • 12. Re: Don't understand zooming a JPEG in Library
                                Bob_Peters Community Member

                                Wow.  I think I just had a sip of water from a fire hydrant!  That's a lot of information to digest.  I wish I had the tools to see what is actually going on in my computers, aside from the obvious problem with the Mac Pro:  It's 5 years old.

                                 

                                Fortunately, I'm not generating 14-bit, uncompressed NEF files.  Mine tend to be a mere 35 to 45 MB.

                                 

                                One final request:  As wizard #155 I need you to invoke your wizardly powers and increase the speed of my computers by a factor of 10

                                 

                                Thanks for takinig the time to generate some ideas and responses.

                                • 13. Re: Don't understand zooming a JPEG in Library
                                  Bob_Peters Community Member

                                  I'm starting to get the message:  D800 NEF files are beasts to render!

                                   

                                  Thanks for the detailed information.

                                  • 14. Re: Don't understand zooming a JPEG in Library
                                    wizard155 Community Member

                                    I'd love to make some suggestions about how to improve your mac but unfortunately there aren't many suggestions which I can make other than the obvious, get a new one The new Intel Core i5 and i7 processors are freakin' fast and I'm looking forward to building my clients beast machine which is going to be running triple NEC 25.5" MultiSyncs, 32Gb of ram, a boatload of storage and SSD boot drive but unfortunately not many people have his budget to spend on his hobby!

                                     

                                    Older hardware can't take advantage of all the new optimisations which are present in the newer versions of the OSX. Plus newer CPUs have smaller cores which translate to less distance to cover to crunch data, more cores, more on die cache, less heat build up and are generally a lot more efficient and quieter.

                                     

                                    If you haven't already, get yourself an Intel SSD which can support the OS and your LR plus cache. Get a newer hard drive to store your photographs onto and of course, make sure you have at least one backup!

                                     

                                    Hard Drives have movable parts in them and they eventually wear out. I've been amazed how much faster a notebook or PC will run if you replace the hdd in them. SSD drives do have a shelf life as well and don't think because they don't "move" that they don't wear out. Intel provides a toolbox software to optimise your SSD drive(s) regularly and to give you a running guide on how well your SSD drive(s) are going. I'm certain other SSD manufacturers have similar tools.

                                     

                                    If you do have an SSD, check whether there are newer firmware updates for it. These firmwares can improve the lifespan of SSDs, better disk optimisations and compatibility. But please please please make sure you have a full backup before attempting a firmware update. If it all goes south, then SSDs are very unforgiving to you and your data in the event of a failure.