5 Replies Latest reply: Mar 16, 2009 10:25 AM by johnrellis RSS

    Speeding up use of the PSE V7 Organizer with large catalogs

    Robert Eckerlin Community Member
      My PSE catalog is relatively large (a little more than 33'000 items/photo Files) and I am using PSE 7 on Windows XP Professional.

      Sometimes the PSE 7 Organizer seems to be reasonably fast; some other times it is desesperately slow. I tried to find out in which circumstances I am happy; and in which other circumstances I am unhappy with its performance .....after making some tests, i found out that the answer is quite simple /obvious; but I had nevertheless the impression that despite the obviousness, it might be useful to a couple of my PSE-user fellows that i describe what I found out.

      On my PC, with my large catalog and in my use-scenarios,
      - the PSE 7 Organizer is most of the time reasonably fast, when I display an Album (i.e. by clicking in the "Album Palette" on an Album) or display the result of a Tag search.
      - And with my large PSE 7 Catalog the PSE Organizer is often much too slow, when displaying (e.g. in Thumbnail view, Import Batch view, or Folder location View) all Photos (without performing a Tag or Album Search).

      Some actions that are fast when displaying the result of an Album or Tag search and slow when displaying all photos of my large catalog include:
      - switching back and forth between Thumbnail View and full screen View
      - expanding and collapsing Version sets
      - removing an item from a Versionset
      - "Mark as Hidden" of an item
      - ....

      I guess (without really knowing) that the explanation for the large difference in speed, is that when displaying all Photos of a large Catalog, PSE rebuilds over and over again the whole Photo-Bin (even though it can only display a limited number of photos on the current screen).

      Therefore, when performing a substantial amount of work on a subset of my photos that are not already part of an Album (and that are not all associated with one Tag or with a couple of tags), i create temporarily an Album, include all my photos in that album and then perform an album search.

      Processing my Photos as part of an Album has for me also an other advantage: I can (when this makes sense for me) order the Photos within that album in the sequence of my choice (e.g. similar photos adjacent to each other).

      ---
      Notes:
      - I do not know if what I described applies also to PSE V6. But i believe to remember that with large catalog PSE V6 and PSE V7 are often substantially faster than previous versions of PSE. This considerable difference in speed justified for me the migration to PSE V6.
      - I know, that my descriptions are not comprehensive and welcome the feedback of other PSE users who also have experience in this area.
        • 1. Re: Speeding up use of the PSE V7 Organizer with large catalogs
          johnrellis CommunityMVP
          Thanks much for sharing that investigation. I havent noticed such a slowth on my 12K catalog when viewing all photos in Thumbnail View as compared to viewing a subset.

          Some thoughts:

          - Have you tried File > Catalog > Repair followed by Optimize? Others have reported that it only makes a modest improvement at most, but its worth doing and reporting the results.

          - Try rebuilding the thumbnail cache from scratch. Contrary to what the documentation says, Repair/Optimize doesnt do that. See step 4 of this FAQ:

          http://www.johnrellis.com/psedbtool/photoshopelements-6-7-faq.htm#_Recreate_Organizer_thum bnails

          (Do this overnight it will take a long time.) Much, perhaps most, of the work that the Organizer must do in displaying photos is retrieving the thumbnails from the cache (which, like the catalog, is a SQLite database). And because the thumbnails are fairly large (compared to the metadata in the catalog database), the thumbnail cache database can get fragmented faster.

          - Have you defragmented your disk? I think the thumbnail cache especially is sensitive to a heavily fragmented disk.

          - A number of us have observed that Folder Location view is much, much slower than Thumbnail view. Is that true for you?

          - Do you have many photos assigned map/GPS locations? Map View can really slow down even moderately sized catalogs.

          - On what kind of drive(s) is your catalog and photos stored? Internal drive in a desktop, internal drive on a laptop, external USB hard drive, network drive, etc?

          - How much memory do you have? The catalog and thumbnail cache databases both benefit from having lots of memory.
          • 2. Re: Speeding up use of the PSE V7 Organizer with large catalogs
            Robert Eckerlin Community Member
            Hello John

            Thanks for your feedback.

            Before answering your questions, I should describe explicitely the following:
            - It was not my intent to "complain"; instead it was my intent to describe something, that on my PC and in my environment is speeding up multiple activities and that could be could be useful to other PSE users having large catalogs.
            - when writing "desesperately slow" in my original post, I did not provide any figures. I realize now, that this could be misleading. For me 10 seconds is "desesperately slow" for activities that I perform often with my large catalog (e.g. switching back and forth between Thumbnail View and Full Screen View). But others could consider 10 seconds to be quite acceptable.

            Lets now answer Your questions:
            - I do not have figures allowing to compare the timing before and after a sequence of "Repair and Optimize". However after a sequence of "Repair and optimize" the time to switch with one single photo from Thumbnail View to Full Screen View is around 2 seconds when displaying the photos of an Album and around 10 seconds when in "Show All" Modus. This is a substantial difference.

            I am curious: what are the timings that you observe on your PC, when switching with one Photo (and just one Photo; when switching with more than one photo the time comparison becomes quite different) from Thumbnail View to Full Screen View?

            - I did not have the opportunity to rebuild all my Thumbnails. But in my particular case, I guess that this would not make a difference (because all my Thumbnails are already built - i pay attention to building the Thumbnails when importing new photos into my catalog).

            - The time difference mentioned above (2 seconds vs 10 seconds) have been measured in a environment with a freshly defragemented disk.

            - I nearly never use "Folder Location View" and can therefore not describe whether "Folder Location view" is also in my environment slower than "Thumbnail view"

            - I do not use map/GPS locations (among other because i have read the posts on this subject; this includes your excellent and very useful posts).

            - My photos and my Catalog are on the internal Disk of my relatively decently configured Desktop PC.

            - on my Windows XP Professional PC, I have 2 Gigabytes of Memory.
            • 3. Re: Speeding up use of the PSE V7 Organizer with large catalogs
              johnrellis CommunityMVP
              > it was my intent to describe something, that on my PC and in my environment is speeding up multiple activities and that could be could be useful to other PSE users having large catalogs.

              Thats the way I interpreted your message, and I think others did too.

              > what are the timings that you observe on your PC, when switching with one Photo (and just one Photo; when switching with more than one photo the time comparison becomes quite different) from Thumbnail View to Full Screen View?

              Similar results to what you observed the fewer the photos showing in Thumbnail view, the faster the switch to Full Screen view. I used Find > Set Date Range to control the number of photos showing:

              # Photos....Switch time in seconds
              2...........2
              150.........2
              2000........3
              4300........5
              5800........7
              9000........7

              Very interesting I never noticed this performance bug before. I just traced the program using a developers tool, and it appears that PSE is accessing the catalog database one or more times for each photo showing in Thumbnail view clearly an inappropriate algorithm in the implementation of the Organizer.

              > I did not have the opportunity to rebuild all my Thumbnails. But in my particular case, I guess that this would not make a difference (because all my Thumbnails are already built - i pay attention to building the Thumbnails when importing new photos into my catalog).

              The thumbnail cache is a database, just like the catalog file. And like any database, it can get fragmented, with data scattered throughout the file and large parts of the file unused. Deleteing the file and rebuilding it will compacted it, just like defragmenting your disk.

              But in this case, my tracing indicates that its the catalog file thats the problem, not the thumbnail cache.

              Ill file this as a bug with Adobe, but based on their past behavior, its unlikely it will get fixed in version 8.
              • 4. Re: Speeding up use of the PSE V7 Organizer with large catalogs
                Robert Eckerlin Community Member
                Thank you John for your explanations and for your excellent problem determination.

                I have the (possibly wrong) impression, that the PSE Organizer, in a lot of different situations, builds over and over again some kind of internal description/map of ***all*** the photos that are part of the "current view" (the "current view" being either "Full Screen View", or "thumbnail View", or "Import Batch View", or "Folder Location View", ....).

                This happens for example when switching from "Thumbnail View" to "Full Screen View", or when switching back from "Fulll Screen View" to "thumbnail View". This happens also often when performing "some" (but, I think not "all") actions that do not switch between views, but that change the set of photos that are part of the current view (e.g. "mark a file as hidden", "remove item from version set").

                For me the most important, is to know that:
                - i can work around these performance problems (e.g. by exploiting Albums or Tags, or by using as you described Date ranges; this reduces the number of photo files that are part of the "current view" and hence improves the performance).
                - and that when using these workarounds, I can achieve quite a reasonable performance with PSE V7 for my large catalogs (there are of course exceptions, like for example geo-tagging that you understand much better than I do).

                And I am quite happy, that PSE V6 and V7 have improved very substantially the performance for large catalogs; and that i do not need to worry anymore that my catalog can become too large to be able to work reasonably with PSE. I am writing this, because I am sometimes reading posts of other disgruntled PSE users, complaining wrongly that new PSE versions did not introduce important improvements. I too get some times disgruntled; but after cooling of, I feel that i need to acknowledge the improvements that have been achieved.

                Cheers, and Thanks again.
                • 5. Re: Speeding up use of the PSE V7 Organizer with large catalogs
                  johnrellis CommunityMVP
                  > I have the (possibly wrong) impression, that the PSE Organizer, in a lot of different situations, builds over and over again some kind of internal description/map of ***all*** the photos that are part of the "current view" (the "current view" being either "Full Screen View", or "thumbnail View", or "Import Batch View", or "Folder Location View", ....).

                  Yes, I think thats right. It keeps the descriptions of the photos stored in the catalog file, which is technically a relational database. Because its relatively slow to read information from a database, PSE caches that information in memory, reading it in only as necessary and discarding it when no longer needed. When it changes the information, it changes the in-memory version and also writes the database.

                  All photo programs designed for large catalogs do something similar. But youve discovered a bug in PSE that is causing it to read in unnecessarily large amounts of information from the database file every time you switch views.