12 Replies Latest reply: Jan 12, 2012 6:46 AM by dj_paige RSS

    LR3 Catalogs and New Year

    hvjkula Community Member

      Just wondering how many users of Lightroom 3 actually create a new Catalog (database) with the start of a New Year?

        • 1. Re: LR3 Catalogs and New Year
          dj_paige Community Member

          Since you asked ... I do not create catalogs by year ... and I consider it a very bad idea to break up your photos into year based catalogs such that you cannot search across years.

          • 2. Re: LR3 Catalogs and New Year
            Hal P Anderson Community Member

            I agree completely with Paige. There is no good reason to do it, and lots of reasons not to.

             

            Hal

            • 3. Re: LR3 Catalogs and New Year
              CSS Simon Community Member

              I agree with the others.  There are enourmous advantages in being able to see and search all images in one catalogue, have arbitrary collections and so on.  There appear to be no performance or reliability issues in very large catalogues. 

               

              If there are very good functional reasons for firewalls between different sets of images (for example to separate personal from professional, to separate different clients), and there is no reason ever to search across them all, then maybe multiple catalogues are a good idea.  I can't see it usually being beneficial for personal use. 

              • 4. Re: LR3 Catalogs and New Year
                b_gossweiler Community Member

                dj_paige wrote:

                 

                Since you asked ... I do not create catalogs by year ... and I consider it a very bad idea to break up your photos into year based catalogs such that you cannot search across years.

                Same here.

                 

                Beat

                • 5. Re: LR3 Catalogs and New Year
                  dj_paige Community Member

                  If there are very good functional reasons for firewalls between different sets of images (for example to separate personal from professional, to separate different clients), and there is no reason ever to search across them all, then maybe multiple catalogues are a good idea.  I can't see it usually being beneficial for personal use. 

                  At the risk of beating this to death (or perhaps in an attempt to clarify further), if you do have groups of photos that need to be kept separate, as CSS Simon has described, multiple catalogs by SUBJECT MATTER makes sense to me. Multiple catalogs by YEAR, as per the original question, still do not make sense to me.

                  • 6. Re: LR3 Catalogs and New Year
                    snahphoto Community Member

                    Hal P Anderson wrote:

                     

                    I agree completely with Paige. There is no good reason to do it, and lots of reasons not to.

                     

                    Hal

                     

                    This is an interesting discussion.

                    I am missing the reasoning, the pros and the cons. Would be nice to learn about the details.

                    As LR-catalogs may grow quickly into 1GB and more, a reasonable approach using multiple catalogs does become an issue for many.

                    • 7. Re: LR3 Catalogs and New Year
                      Hal P Anderson Community Member

                      On even a .5 TB drive 1GB is quite small. I know people with single catalogues comprised of several hundred thousand images. As catalogues grow into the 50+GB range, you have to plan ahead to do backups, but the advantage of keeping everything in one catalogue outweighs that.

                       

                      Mainly, the problems you'll have involve searching for images or sets of related images.

                       

                      How do you find all the photos of Uncle Harry if he's scattered across decades of single-year catalogues? Typically, you'll want to have all the pictures together in the Grid view, but if they are in different catalogues, you can't without a great deal of hassle.

                       

                      For that matter, how do you find that picture of him that you took in his house that summer afternoon around 15 years ago? Basicallly, you have to open a series of catalogues because you probably aren't going to remember the exact year. I know I couldn't.

                       

                      Even the task of bringing all your five-star images together so you can print them or send them to a friend can't be done if you spread your work across several catalogues.

                       

                      Lightroom doesn't appear to slow down with big catalogues. If you want to find images easily, you need to keyword and perhaps rate all of them, but that would be true even if you have a catalogue per year.

                       

                      Speaking of keywords, they're kept in the catalogue, and if you have more than one catalogue, you run the risk of your keywords not matching, due to spelling errors or deciding on different terms for the same thing. One year it's cantaloupe and the next it's "musk melon". So if you misspelled Harry in one of the catalogues, when you search for him there, you'll not find him and just assume he's not there, that year. But he's there, and Murphy says that his best picture is actually there, but with the wrong keyword.

                       

                      Bad idea. Don't do it.

                       

                      Hal

                      • 8. Re: LR3 Catalogs and New Year
                        vbnut Community Member

                        I couldn't disagree more.

                         

                        I have multiple catalogs to serve different purposes.

                         

                        I have one "processing" catalog for images I have imported, but haven't complete processing yet.  It typically contains my backlog of 1000-2000 (but currently over 4000) images.  I backup this catalog every time I quit LR, and it takes a few seconds.

                         

                        I have a catalog for each year and they contain between 3000 and 10000 images each.  I also backup these catalogs every time I quit LR,and it typically takes 20-30 seconds.  When I finish processing the images from an event (in the "processing" catalog) and export them as needed, I delete the rejects, switch to the catalog for the appropriate year and do an "import from catalog" to import those images into the yearly catalog, switch back to the "processing" catalog and remove those images from that catalog.  I only use the yearly catalogs when I need to do specific tasks (e.g. select my best wildlife photos of the year to create a calendar).

                         

                        I have a "all years" catalog created by using "import from catalog" using my yearly catalogs.  It contains ~40000 images.  I have this catalog set to backup weekly, and it takes over 200 seconds, but fortunately I rarely use this catalog so I don't pay the cost very often.

                         

                        I spend most of my time in my "processing" catalog working on my backlog of images.  If I was to do everything in a single catalog that contained all my images I would waste 5-10 minutes a day waiting for LR to check the catalog, back it up and then optimize it.  I would have to wait until this is complete so that I can logout of the computer.

                         

                        On the unusual occasions that I need to deal with an entire year of images or the very rare situation where I need to deal with all my images, I can switch the the appropriate catalog, but fortunately I don't need to do that very often.

                         

                        FWIW, I haven't created my 2012 catalog yet because I'm still busy processing the images I took in October 2011 .

                        • 9. Re: LR3 Catalogs and New Year
                          dj_paige Community Member

                          You spend a lot of time moving photos from catalog to catalog, waiting for the next catalog to open, and managing catalogs. I don't spend my time on those tasks (and I suspect Hal doesn't either) because I don't do those tasks. There are tools within Lightroom to handle every situation you mentioned. Lightroom has many built-in organizational tools, and you have chosen to use the most unwieldy of those tools, the catalog, to perform these tasks.

                           

                          The one drawback of a large catalog (takes longer to backup) in your case, is simply split up into backing up many smaller catalogs, I don't see any gain in time the way you do things (and I don't particularly care if LR takes 3-5 minutes to make a backup of my catalog, because I can go and do something else productive while it does that). To avoid a task that the computer does unattended (backup) where you can go do something else, you have substituted many smaller tasks that cannot be done unattended.

                          • 10. Re: LR3 Catalogs and New Year
                            hvjkula Community Member

                            Many thanks to all for your responses.

                            • 11. Re: LR3 Catalogs and New Year
                              snahphoto Community Member

                              I am very glad receiving your numerous responses about the many options to handle single or multiple Lightroom catalogs.

                              It appears to me that back-up time is one critical argument, production efficiency another and catalaog size as well, but much lower positioned on a scale 1-10.

                               

                              What about the principle of 'NOT putting all eggs in the same basket'? Isn't this a universal law advisable to follow? Financial advisors as well as security experts attempt to strictly follow this universal law. So why shouldn't lightroom users follow?

                               

                              What about in the event of hardware limitation, hardware failure or data loss? Hardware limitation can be overcome by simply adding diskspace or RAMemory whereas the only weapon against HW-failure and data loss is an effective and proper back-up strategy.

                               

                              Technically there is a file ending '<catalog-name>.lrcat' (the actual catalog) and then there is the folder named '<catalog-name>.lrdata'.

                              Both instanances depend on eachother and are placed in their common folder named after the catalog itself.

                               

                              As I understand the Back-up of a LR-catalog creates an optimized and clean copy of '<catalog-name>.lrcat'. But it leaves out the folder '<catalog-name>.lrdata' containing all rendered image previews. This folder appears to be essential in regards of the criteria 'production efficiency' since performance of LR heavily depends on the settings for the catalog 'preview cache'. And since LR creates one folder per image the OS requires lots of RAM serving large catalogs.

                               

                              There are several space-saver options:

                              - For the catalog '<catalog-name>.lrcat' the biggest space-saver may be the removal of all 'edits', if desired. (tip: sort images on edit count, write Metadata to XMP, delete History).

                              - Space-saver options for folder '<catalog-name>.lrdata' are defined by settings for the preview cache.

                               

                              It seems to me one has to find the balance between efficiency and size in regards of available hardware.

                               

                              So far I learned that for efficient production the catalog (folder and file) should be positioned on the fastest disk available or maybe better on a RAID5-subsystem with a high bandwidth.

                              Whereas the back-up of the catalog file and folder should be placed on at least one other drive.

                               

                              I assume that there are other aspects worth taking into consideration in regards to managing Lightroom catalog(s).

                              What are your thoughts?

                              • 12. Re: LR3 Catalogs and New Year
                                dj_paige Community Member
                                What about the principle of 'NOT putting all eggs in the same basket'? Isn't this a universal law advisable to follow? Financial advisors as well as security experts attempt to strictly follow this universal law. So why shouldn't lightroom users follow?

                                I am not aware of a compelling reason for this principle to apply to Lightroom, or to digital photography in general, or to computer files in general. For digital files, there is a concept of "backup", which creates an exact duplicate in another location; there is no equivalent concept of "backup" for actual eggs.

                                 

                                What about in the event of hardware limitation, hardware failure or data loss? Hardware limitation can be overcome by simply adding diskspace or RAMemory whereas the only weapon against HW-failure and data loss is an effective and proper back-up strategy.

                                As you explain, there are effective strategies to prevent loss of data.

                                 

                                Technically there is a file ending '<catalog-name>.lrcat' (the actual catalog) and then there is the folder named '<catalog-name>.lrdata'.

                                Both instanances depend on eachother and are placed in their common folder named after the catalog itself.

                                I'm going to have to disagree here with the part "both instances depend on eachother [sic]". If the catalog accidentally were to move to a new location, you could still open it by double-clicking on it, and it would work properly.

                                 

                                As I understand the Back-up of a LR-catalog creates an optimized and clean copy of '<catalog-name>.lrcat'. But it leaves out the folder '<catalog-name>.lrdata' containing all rendered image previews. This folder appears to be essential in regards of the criteria 'production efficiency' since performance of LR heavily depends on the settings for the catalog 'preview cache'. And since LR creates one folder per image the OS requires lots of RAM serving large catalogs.

                                Lightroom will recreate your cache folders of rendered image previews if they are not found in the expected location. Nothing is lost (other than some computing time) if you have to recreate them. Perhaps this time to recreate the previews is crucial to you; it is not to me.

                                 

                                I disagree with "since LR creates one folder per image the OS requires lots of RAM serving large catalogs". I don't think "lots of RAM" is required for large catalogs; a large catalog or a small catalog will operate at approximately the same speed for equal amounts of RAM, except when editing (develop module). Large catalogs really do not require more RAM.

                                 


                                There are several space-saver options:

                                - For the catalog '<catalog-name>.lrcat' the biggest space-saver may be the removal of all 'edits', if desired. (tip: sort images on edit count, write Metadata to XMP, delete History).

                                - Space-saver options for folder '<catalog-name>.lrdata' are defined by settings for the preview cache.

                                Maybe this is "efficient" for you; to me it seems incredibly inefficient and defeats the entire purpose of a catalog. It is inefficient because you are replacing automatic operations that Lightroom performs to store your edits with manual operations of removing edits. I cannot see a single reason how this is "more efficient", nor am I convinced that it saves space. Earlier, you were arguing for "production efficiency", which to me means that you don't want to lose time if something happened to your previews; now you are arguing to save space; normally "production efficiency" and saving space are diametrically opposed, to get one you have to sacrifice the other.

                                 

                                You have presented a variety of arguments that all could be used towards the conclusion that you might want somehow you might want to do something different with your catalog(s). I find none of your arguments compelling. I also think that Adobe has put a lot of thought into the design of Lightroom and it seems to have optimized the experience of using Lightroom for many people. Of course, it can't optimize for every person's needs, there will obviously be individuals whose needs are different and would benefit if Lightroom operated in a different fashion ... but, I am wondering if you have given Lightroom a full workout, using a single catalog, to see if it meets your needs. Your arguments sound hypothetical, in that you think you don't like the setup, rather than you tried something and it doesn't work for you. Have you really used Lightroom for a while to see if it meets your needs?

                                 

                                You are way off topic here, this has nothing to do with the original topic of Catalogs by Year, you would be better off discussing this in a new thread, and I would be happy to continue the discussion there.