13 Replies Latest reply on Dec 4, 2018 8:44 AM by whsprague

    Clearing the Library

    dunker56 Level 1

      In an effort to speed up my LR experience I am curious if on my system a clean library would assist, therefore I need some exporting catalog or clean up advice.


      I already have a pretty odd LR setup with how I manage my HDDs and stuff, 80% of my library doesn't work anyway because the original image file is no longer there (moved to another hdd).


      My library says I have around 3427 images (that's what it says on my most recent import, a transparent '3427' kinda behind the thumbnail). I do use the Library feature, but only for a few sessions at a time. Once my internal SSD's get to full I migrate the RAWs and Exports to by backup Quaddock (holds multiple HDD's and is not left on all day as it is noisy and drains unnecessary power).


      Perhaps it will help explain my workflow. It is important I don't want advice in this specific regard, I deliberately work like this to avoid having to constantly have 8TB's worth of HDD's whirring away for 90% of the time unnecessarily, I enjoy the low noise performance of my Skull Canyon NUC (which has an SD Card slot ).


      • Take photos on camera (writes to two cards simultaneously).
      • Eject one SD Card from camera, put in the NUC
      • Use Faststone Image Viewer to browse the RAW files quickly, tagging the keepers, I find this far far more fluid than importing a massive volume of files in LR and dealing with the cull at that point
      • Once I have tagged my keepers I use a feature in Faststone that moves those Files/Images to a new location (ie from the SD card to a Folder on the C:)
      • Eject SD Card from the NUC, put back in camera
      • Using Faststone Image viewer again, fire up the first image I want to work on in LR, I simply click 'E' and it opens the file in LR (prompts the Import).
      • I import, edit and Export the image
      • However, the C: is only a 250gb SSD, therefore around about every week to fortnight I need to Move both the RAW files and the Exports to my HDDs, to give me back 'working space' on the SSD. This however is much more preferable to me as it means I only have to toggle the HDD's once a week/fortnight for a few minutes. Bypassing the SDD and instead use the larger HDD from the outset not a viable option to me, it means I would have to toggle on my external HDD's daily, it completely negates the idea and premise against the purchase of the low powered/quiet NUC in the first place.


      So that's what I'm left with, a LR library of hundreds of images, of which I still have all the files but the library is not able to find the files anymore as they've been moved.


      I am hesitant to delete or wipe the library because it could be one day I want to go back to an edit rather than start from scratch again, I figure there must be a way to tell LR to also check 'here' (and make it point to my HDD where the RAW file will now be), and then proceed with the edit, or failing that manually find the image on the HDD and drag it back to it's original location on the SSD so LR can 'see it' again.


      Truth be told I have not had to do either of those two things yet, however I like to be safe rather than sorry.


      I mustn't be the only user using LR like this. People whom use it on laptops or tablets with limited internal SSD space, they must also have 'broken' libraries?


      So two things;


      1) A library of 2000+ is silly anyway, so many images, I can't find what I'm looking for easily, and I get a sluggish 'searching' experience

      2) I'm hoping that clearing the library, or at least canning a certain amount to leaving the last 200 images or so still there will lead to a quicker navigation and less sluggish LR experience. I'm sure if my PC had proper processing power this might not be such an issue.


      So how do I empty my library in such a way that should I want to work on an old image I can 'import' my old library (catalog?) and then either point it to where the file now exists or manually put the file back to the SSD?



        • 1. Re: Clearing the Library
          whsprague Adobe Community Professional

          By ignoring the basic tenets in how Lightroom works, there is no way anyone in a forum environment can efficiently get your mess straightened out.  Stop using Lightroom!  It is not for you!  From Fastone, edit in Photoshop that uses the same Lightroom tools in ACR.  It will save your RAW work in .XMP sidecar files. 


          If you insist on using Lightroom, do two things.

          1.  Start using Lightroom, not the Windows Explore, to move you files off you HDD.

          2.  Start a process where you gradually reconnect LR to your moved files.


          If you don't want to do #2, abandon your old catalog and start a new one.

          1 person found this helpful
          • 2. Re: Clearing the Library
            dunker56 Level 1

            I use synchronisation of LR images a lot, so it most definitely is for me, as well as using a fair few paid Presets that I find incredibly useful... but I digress...


            1) When I do my Windows Explore moving of files, it's because I am moving other files as well, some musical etc. Using LR to move some of the library files might work, but can it move the export files as well? It seemed to me simpler to use a 3rd party synchronisation software when doing the migrating of files from the SSD to HDD, a one click and it's moving various files. It's made even better because once that sync is complete I start another sync which then makes a back up of the back up . Two clicks, done.
            So for that reason I am not overly keen to start getting LR to do the moving of files, it's an additional weekly/fortnightly step I don't want to make.


            2) I don't want to start a process where I gradually redirect LR to my moved files. I have a library of 2400 files (from this installation alone), I have WAY more files that than, and not ONCE have I actually needed to go back and tinker with an edit from 6-24 months ago. BUT... I would like to be able to, should the situation ever arise (rather than working on that file from scratch). Starting a process whereby I gradually reconnect LR to my moved files sounds awfully like toggling on the HDD's way too often (remember, I like having it off for 95% of the time, turning it on for only half an hour or so once a week/fortnight is preferable).


            3) Abandoning the catalog sounds like losing editing work of 2400 files, is this what you're actually suggesting?


            Can I not turn on my HDD's now, export the library/catalog to the HDD, perhaps not all of it but perhaps 90% of it and store it on the HDD? Then... should I ever need to go back to an edit I can import that catalog back into LR, and locate the file either manually or tell LR to scan 'this folder <points to HDD where the RAW files now live>' let it scan, do it's thing and then edit from where I last left off?


            Essentially I use the Library feature a lot, but only the past 50-100 images typically. Not 2400. I have a feeling clearing out the library might gain some better performance, but then again it might not, but I think I would find images a little easier than dragging the side scroll bar up and down so much.

            • 3. Re: Clearing the Library
              WobertC Adobe Community Professional

              +1 for the reply from whsprague

              You are wasting your time and energy with your Lightroom methods!

              Understand how Lightroom works.

              Start with the free eBook  "Lightroom Classic CC Quick Start"- https://www.lightroomqueen.com/blog/

              Buy a Lightroom book, read it cover to cover.


              Or as stated- "Stop using Lightroom!  It is not for you!"

              • 4. Re: Clearing the Library
                dunker56 Level 1

                Does abandoning this current catalog and starting a new one mean I lose the old catalog and the edits? I'm in the middle of editing a session, can I create a New Catalog and move some of the images from the old catalog to the new one?

                • 5. Re: Clearing the Library
                  WobertC Adobe Community Professional

                  Does abandoning this current catalog and starting a new one mean I lose the old catalog and the edits?

                  Effectively- Yes.

                  Images are not "IN" Lightroom catalogs, so you cannot "move" images from old catalog to new one. (Other than by combining the previews using a catalog merge function- {Import from another Catalog}..)


                  My suggestions-

                  Use your current Catalog (as small as it is!),

                  Re-link as many of the previews for 'missing' files that you can so you retain their edits.

                  How to find missing photos in Photoshop Lightroom Classic CC


                  If it becomes impossible to re-link 'missing' files then it is possibly best to "Remove" them from the Catalog (and lose edits for these).

                  When you have re-linked as many as possible, do an "Import" of ALL your camera photos anywhere in your computer and external drives (using the ADD option).


                  Do not move anything anywhere until you have re-established the complete database (catalog) of ALL your photos (except backups).

                  Do not use the Folder 'Synchronize' function as this can 'remove' missing photos from the catalog un-intentionally.

                  General advice- Never use Windows File Explorer to Move, Rename, Delete, files or folders. (there are a few exceptions).


                  You must use Lightroom to reference every photo you have, whether you "need" it or not. There are ways to 'limit' your views in the Catalog Library so you only see recent imports.

                  • 6. Re: Clearing the Library
                    dunker56 Level 1

                    During the time taken to get the responses I was in Live Chat with an adobe rep.


                    We created a File>New Catalog and named it something appropriate, and I was even able to select 24 images from the last catalog and port them into the new one.


                    I now have a catalog that has 24 images, an incredibly faster load time and user experience (night and day difference).

                    My old catalog (the only one I've ever had or used) had actually over 3400 images in it (not 2400 I thought, I'm number dyslexic).


                    I cannot possibly go through that old catalog and find missing files etc, I imagine 95% are missing anyway as the majority of files have been moved to the external HDD. I simply don't have the time to do that.


                    You wrote that effectively by creating a new catalog you lose the old edits... this is not correct though is it? What's stopping me at any time from File>Open Catalog, selecting old catalog, and then selecting the image I want to revisit, getting the 'missing file' notification and either manually finding the RAW file from my HDD and putting it back to where the catalog wants it to be, or tell the catalog to 'check out this location for the missing file' and wait a bit?


                    I repeat... in my 2yrs of shooting I have yet to actually want to revisit a file for touch ups, it's not a scenario I am likely to face, but just in case it would be nice to be able to do that. How is having multiple catalogs effectively stopping me from doing this exactly?


                    Where I find 'library' and 'catalogs' handy is when working in a single session/shoot. I edit a shot, choose the style, edit the presets etc, and then for continuation/consistency/speed of work flow purposes use the Sync feature. But once I have done those edits, and a week or two passes, I am unlikely to really revisit those images for tweaks again.

                    • 7. Re: Clearing the Library
                      WobertC Adobe Community Professional

                      What's stopping me at any time from File>Open Catalog, selecting old catalog, and then selecting the image I want to revisit, getting the 'missing file' notification and either manually finding the RAW file from my HDD and putting it back to where the catalog wants it to be, or tell the catalog to 'check out this location for the missing file' and wait a bit?

                      Yes you can open the 'old' catalog. Nothing stopping you.

                      You can "put (photos) back to where the 'old' catalog wants it to be". BUT, and the problem it creates- Now the 'New' Catalog has lost the file location and it is shown as 'Missing' in the 'new' catalog, because you moved the file.  Another strong argument for only having ONE catalog.


                      I am unlikely to really revisit those images for tweaks again.

                      My son wanted old photos of him as a child for a Birthday event- I had him keyworded in files back to 2004- so easy to find with one click in the Keyword list panel!  And as for "tweaks"- you might be amazed how the more recent versions of software can do a better job of edits.


                      One catalog- all photos.! that's my mantra for Lightroom.

                      • 8. Re: Clearing the Library
                        dunker56 Level 1

                        Thanks for that.


                        I have a feeling LR is just set up for certain systems. A nice powerful desktop with multiple HDD trays and yer set. Or.. you move to cloud based LR.


                        A  NUC with 250gb SSD and and Australia's ADSL2+ net speeds (5mb/s dl, 0.8mb/s ul) makes cloud based options null void .


                        I used to own a higher powered desktop machine, but when you get months of 38-45 degree Celsius days powering a lot of tech for times when you really only access certain disks is not fun in terms of additional heat generated. Moving to the NUC has been mostly a positive change, the low noise, low heat has been a welcome relief. But I've had to 'juggle' files and that's come at a price, but mostly only this issue gas been the worst outcome of all.


                        I understood that LR can import direct from a sd card, I tried that. I understand LR can change preview modes so it lowers the detail and increases speed of import, I understand LR can handle the culling procedure and I tried that also.


                        We all shoot differently, in the end I found FastStone Image to being the least fussy fastest way of managing my work flow. I tried to leave adobe behind, tried Affinity etc, but in the end valued the Sync aspect of LR too much to leave it.
                        I also value Presets and the sync aspect of that, something that is seldom featured in other software (perhaps C1 only). I do use PS for masking, layers and other plugins, but where possible staying within LR is quickest.


                        I tag most of my work on flickr (4000 images) so I can locate older files. I can edit them even though they would be jpgs. Nicer to tinker with the RAW for sure.


                        You must have a nice PC, because moving from a catalog of 3400 to 24 has made a significant difference to the work/interaction speeds I'm getting.

                        • 9. Re: Clearing the Library
                          WobertC Adobe Community Professional

                          Referring back to your first post in this thread I see your situation thus-

                          You have the NUC, and for large storage- the quaddock (External)


                          You should continue to Import new photos into ONE Lightroom catalog kept in the NUC.

                          You should connect the quaddock and Import ALL your photos into the Catalog (with "ADD" option).


                          Now when the NUC storage is filling up- Use the Lightroom Folder panel to move 'old' photos to the external storage, but do NOT remove them from the catalog. So even if you turn off the 'external' you will still see all the previews in the catalog even though they may be marked as 'missing'.


                          If you want to go back to editing an 'old' photo it will still be in the Catalog (with edits) and only needs the external device reconnected.


                          In fact you could store ALL your photos external to the NUC and only create Smart Previews for a limited number of photos that you want to edit without the external device connected.


                          My Dell XPX13 250Gb SSD puts me in a similar situation to you (but I do have desktop also ~70,000 images) My laptop catalog has 7889 photos shown in the catalog. 785 of these are 'missing' because they are stored on an external portable HDD currently disconnected, but I have these as Smart Previews so I can see them in the Library and edit them at any time.


                          So- One Catalog, All photos in the Catalog. Use Lightroom to its full capability of features for storage options.


                          Also a Aussie- in Queensland's heat. Thankfully no fires in my location. About to be connected to the NBN on Monday, so I will see if it is better than ADSL, but, like you, still not a keen user of the 'Cloud- Lr-CC'.

                          1 person found this helpful
                          • 10. Re: Clearing the Library
                            dunker56 Level 1

                            Ok thanks for that, detailed and great approach. What I am now concerned about is performance.


                            At the time I had actually 3400 images in my Catalog. Performance was sluggish, even when not in Develop mode but just from the import of new images, selecting images from the Library, selecting multiple images for sync purposes and all stuff like that, it was sluggish.


                            Since moving to a new catalog with just 24 photos, it's like a fresh install again
                            So I am hesitant to go back to having one catalog as although the fix your presented would work help older files etc, I'm wary it will return the sluggish LR interface and actually possibly make it even worse (by finding more images).


                            So... two things;


                            1) Is there a way to build the catalog back up but somehow have things working more fluid with a large catalog?


                            2) When you do the 'ADD' option erase old edits at all?



                            • 11. Re: Clearing the Library
                              whsprague Adobe Community Professional

                              I don't know how to replicate a catalog with few thousand disconnected images.  I can't confirm "sluggish performance".


                              My catalog with about 35,000 connected images runs well.  Very well!


                              Your concept of erasing old edits while importing new files only underlines that you are not in need of the Library functions.  You are trying to outsmart Lightroom.  It just does not work the way you are trying to make it work. 


                              Again, consider abandoning Lightroom.  Instead,  use Photoshop with the ACR function.  Manage your photo organization with Bridge. 

                              • 12. Re: Clearing the Library
                                dunker56 Level 1

                                Well we all have different financial means, I'd upgrade the NUC and sort storage issues out then make use of the Library feature fully (obviously) but that's not a viable option currently.


                                Correct me if I am wrong but the only way to 'Sync' (edit) settings across images is with the Library tool? When in Develop module for example, you either expand the catalog from being a film strip at the bottom to being on the second monitor, and then using Ctrl key select the images you want to sync etc. This is a highly crucial feature I need for my work, it's a reason I haven't left Adobe in favour of other editors as they all seem to miss the synchronisation aspect.
                                Can Syncing of edited image settings be completed in ACR? Furthermore I use various Presets such as VSCO and RNI, can they too be accessed from ACR or only LR?


                                These days I am trying to restrict my access to PS as whatever changes I make to an image there can't be synced when back to LR etc.


                                It's quite simple really, poor man, poor system, trying to find a balance between using the features I need whilst keeping optimisation at it's maximum, until inheritance money kicks in

                                • 13. Re: Clearing the Library
                                  whsprague Adobe Community Professional

                                  Since you rely on the Sync feature, this is what I might do in your situation.   (This has become a long topic, so I apologize in advance if I'm stepping on other posters' toes.)


                                  The goal is to convert LR Classic to a (1) batch photo editor system with (2) minimal files in the catalog and (3) not lose any work.   Your new catalog will have only your "active" files in it.


                                  First.... In Edit > Catalog Settings, check the box for "Automatically write changes Into XMP".  This creates a tiny "sidecar" file with the same develop setting results as are recorded in the catalog as you do your editing. 


                                  Second... Import the files you want to work on into a clean catalog.  Work on them as you normally would.  When finished, select and delete them from the catalog.  Note that the delete process always gives you a choice between deleting from the computer completely or just removing from the catalog.  Delete only from the catalog. 


                                  Third.... Move your files WITH their new little companion XMP file anywhere you want with any tool you want.


                                  Fourth....If you ever want to edit those removed files again, use the Import dialog.  LR will see and read the XMP files and you will return to having the adjustments where you left them when you deleted from the catalog.


                                  Regarding your old catalog with all the broken links...  You can keep it if you ever want to try to relink a file.  You could make a copy of it, open it up and clean out all the broken links.  Or, you could abandon it as an unrecoverable mess.

                                  1 person found this helpful