7 Replies Latest reply on Jul 21, 2016 1:04 PM by ManiacJoe

    Want to synch all images on my NAS - not for use by LR catalog but as duplicates for emergency

    jonathan+7007 Level 1

      Context and environment:

      Professional photographer

      Win7 Pro 64-bit tower production computer

      Ivy Bridge, about to build Skylake tower.

      32Gig RAM

      Gigabit network with NAS in another part of the house

      150,000 images, xmp files separate.

      Lightroom CC 2015.6 and Photoshop

      Lightroom is my DAM system/software

      LR images are stored on my desk on a WD MyBook Duo 4TB RAID array which communicates with the motherboard by USB3.

      Goal:

      Everyday that I make changes to a few files or a 1,000 images I want the NAS to be synchronizing those files that changed to stash a second copy up at the opposite end of the house. Benefit: Synchronization and Separation.

      Current Storage gear on the network:

      Synology 215j with 5.5TB capacity (two WD 6TB Red drives in RAID)

      WD MyBook 4TB

       

      Use of a laptop while away from the production machine and the database -- not a high priority.

       

      Ok, now I can ask the question:

      Synology tells me that they think their database and motherboards aren't suited to the model I wanted to use: the 215j would be rolling up all changes. They aren't Lightroom experts, though. I thought the xmp files were what would be changing but these sheer size of the first "uptake" of all the images and their xmp files  has thrown the synch off twice. Now Synology Support is blaming the scope of the task: nearly 300,000 files that Windows Explorer can find inside the top-level "Lightroom" folder. They also suggest that a day's editing might make too many files to synch. Further, the Tier Two person I m corresponding with tells me that a bigger faster (more expensive) Synology box isn't going to work better.

       

      Just copying folders every Friday is not a viable option.

       

      I already have CrashPlan storing files way off in a server farm somewhere, so the far-off-site variation is already taken care of.

       

      What solution to this localized synch are Lightroom experts pursuing?

        • 1. Re: Want to synch all images on my NAS - not for use by LR catalog but as duplicates for emergency
          dj_paige Level 9

          Everyday that I make changes to a few files or a 1,000 images I want the NAS to be synchronizing those files that changed to stash a second copy up at the opposite end of the house.

          But when you edit a photo in Lightroom, the photo files themselves are not changed at all. So there's no need to update the photo files on the NAS. As long as a copy of the photo ORIGINAL files are on your computer and on the NAS, and you have a backup of the catalog file somewhere, you have a legitimate backup.

           

          So with that in mind, what you want to do somehow is to make regular and automated backups of the catalog file (if desired, the catalog backups could go on the NAS).

           

          So if I'm understanding you properly, only one file needs to be transferred/created on the NAS, and that is the catalog backup file.

          • 2. Re: Want to synch all images on my NAS - not for use by LR catalog but as duplicates for emergency
            jonathan+7007 Level 1

            Mr. Paige-

            Thank you for offering that suggestion, and yes, I know the images in my Lightroom topmost folder, once they have been imported and cataloged will never change. There are two steps to the creation of the backup tool I wanted. (1. create a synchronized copy of the whole Lightroom image and xmp folder, then 2. as edits occurred and new images were added those changes in content would be synchronized with no intervention on my part.)

             

            A Synology tech resource told me I would never get through the first step. Too many files.

             

            So, I am still open to suggestions about how to accomplish the level of safety/backup I want.

             

            I back up the catalog every time Lightroom exits. I do copy those catalog backup files from time to time to my two NAS boxes. As another form of back up I set Lightroom to copy the images to a a spinning hard drive inside the production machine during Import. That's not as well protected as the NAS in the other ed of the house. And these Import folder copies are a jumble of files, too.

             

            Here's a way forward that might be a reasonable compromise tactic. I copy older Lightroom sub-folders (2011, maybe 2012, and older) into an "archive". Then I set a synchronization "task" (Synology's term) for the Synology 215j to keep real-time track of 2012 or 2013 through the current year -- maybe even push that further -- 2014 -- and newer... whatever the software will do without stopping mid-process as it has been doing. More recent files get the most "action", and the last 3 months the lion's share of edits at any point in time.

             

            I see one problem here. The catalog file does NOT know about an static archive that is separated. The catalog file has everything recorded: all the folders.

             

            Maybe it's better to split the whole set of pictures into two or three catalogs for manageable size just for backup. Yuk, I'd give up the ability search the whole image set.

             

             

            Any ideas considered,

            • 3. Re: Want to synch all images on my NAS - not for use by LR catalog but as duplicates for emergency
              JimHess Adobe Community Professional & MVP

              In my opinion, you are trying to make this more difficult than it needs to be. If you have a backup of your images (and you should, of course) and you backup the catalog, that's really all you need. The images are backed up, and the catalog contains all the work you've done. It doesn't matter if you are a professional photographer with lots of images. This backup procedure is really all you need. If you want to make it more difficult then go ahead and do so.

              • 4. Re: Want to synch all images on my NAS - not for use by LR catalog but as duplicates for emergency
                dj_paige Level 9

                I have to agree with Jim Hess, I would say the exact same thing, you are making this more difficult than it needs to be.

                A Synology tech resource told me I would never get through the first step. Too many files.

                Then you have chosen the wrong tool if you can't even do the first steps. Any NAS/RAID/backup solution ought to be able to handle the first step.

                • 5. Re: Want to synch all images on my NAS - not for use by LR catalog but as duplicates for emergency
                  jonathan+7007 Level 1

                  Yes, I take steps to back up the raw files and the catalogs. This newer "synch" procedure was my attempt to add an automatic(background) and closer-to-real-time process to the backup. Both respondents above believe that the xmp files do not need to be backed up. I thought the edits referenced in the catalog file needed the xmp files to be implemented and therefore saved in a backup. So it's off to do more research.

                   

                  it's possible that another NAS operating software can do what Synology cannot.

                  • 6. Re: Want to synch all images on my NAS - not for use by LR catalog but as duplicates for emergency
                    dj_paige Level 9

                    I thought the edits referenced in the catalog file needed the xmp files to be implemented

                    Incorrect. Everything in Lightroom works perfectly well without xmp files, in fact that is the default to not have xmp files. Even when xmp files are present, Lightroom does not use them unless the user specifically tells Lightroom to do so.

                     

                    Reading between the lines, it seems like your incorrect understanding of xmp files is driving your rather complicated backup plans; and the correct understanding could enable the much simpler backup plans that have been mentioned by myself and JimHess.

                    • 7. Re: Want to synch all images on my NAS - not for use by LR catalog but as duplicates for emergency
                      ManiacJoe Adobe Community Professional

                      After reading this thread, you may be overthinking this.

                       

                      LR and your pictures are completely implemented at the simple file level. Because of this, any file copy programs will get you to be exactly where you want to be.

                       

                      My win7 box uses XCOPY and ROBOCOPY in batch files to copy all the new and updated files to the backup drives. I can run the batch files manually as I need. I can use Task Scheduler to run them on a schedule if I desire.