Skip navigation
Currently Being Moderated

backing up takes almost an hour

Aug 19, 2010 11:46 PM

It routinely takes almost an hour to back up my main LR 3 catalog, which has over 300,000 images in it. Is that normal? It's pretty annoying having to wait for that when I shut down my computer at night. I'm optimizing the catalog right now, but IIRC I've optimized before and still had the long wait.

 

I'm on a Mac Pro dual core 2.66 with 5GB ram, OS X 10.6.4.

 
Replies
  • Currently Being Moderated
    Aug 20, 2010 12:34 AM   in reply to jimtron

    pain isnt it? long backups and testing integrity and then optimizing,

    Try unchecking the test and optimize for week days, and only doing that once a week.

    catalog.jpg

     

    This speeds up my tiny 79,000 image library catalog substantially.

     

    Or finish work an hour earlier to allow the back up time whilst you have a mug of ovaltine or horlicks or hot chocolate as a nightcap.

    Or dont back up at night, rather turn on the computer before breakfast,open and then close lightroom and allow it to back up whilst you have coffee and croissants, read the paper and watch the dogs playing in the garden.

    This is not a workaround, rather you have a HUGE catalog that takes time, this is more time management

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 20, 2010 3:49 PM   in reply to jimtron

    jimtron wrote:

     

    Thanks for the reply, I'll give that a try. This is probably a stupid question, but where are those backup prefs? I see in preferences where you can choose backup frequency, but I don't see the preference pane you attached.

     

    That's the dialogue box that comes up when the backup is about to start. Not a pref.

     

    Hal

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 26, 2010 11:25 PM   in reply to jimtron

    how large is your database?

     

    I'd do one more overnight back up and then contact Adobe and see what they have to say on the matter.

     

    Other checks -

    hard drive space?

    fragmentation?

    OS windows or Mac?

     

    Other than that, 5 hours is crazy, I  get peeved with 10 mins, but my catalogue is only 80,000 images and 1,000 are virtual images...

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 26, 2010 11:36 PM   in reply to jimtron

    Youno issue with mac and fragmentation, its a PC thing.

     

     

    300,00 images, how many Gb is the file?

    Click Lightroom - Catalog settings and you get this dialog that indicates the size in Gb of your catalog

     

    Screen shot 2010-08-27 at 08.31.38.png

     

    Backing up does seem to slow things, but 6 hours is not so lekker.

     

    I'd talk to Adobe about this,  try and PM Melissa Gaul - on the lightrom team

    http://forums.adobe.com/people/MelissaJ.G.

     

    Sorry man, well out of my domain there.

    Good luck

     
    |
    Mark as:
  • Currently Being Moderated
    Dec 15, 2010 5:19 PM   in reply to jimtron

    Yes, back-up hangs up sometimes.

    I have about 25,000 images in LR.

    I made these observations:

    1) The progress bar hangs when it is almost at the end. If I suspect it hangs I hit the enter key, a dialog box comes up saying that LR was not able to finish the back-up. Mostly it is fine on the second try.

    2) The hang-ups occur when I have done a lot of work and not done a back-up for a few days. When I backup at least once a day, hang-ups don't occur.

    3) I have never timed it exactly. I guess my back-up takes about 10 min.

     

    Skipping optimization of the catalog is certainly an option when the back-ups take too much time.

    But I wonder if it's not worth the wait. I have a hunch that a not optimized catalog might slow LR down by quite a bit in its work.

    WW

     
    |
    Mark as:
  • Currently Being Moderated
    Dec 15, 2010 5:29 PM   in reply to web-weaver

    I'm on a Mac with a decent sized catalogue -- 100K images or so (though I don't know the actual size at the moment.) Backing up takes only minutes.

     

    5-6 hours for a few GB of data is too long, and suggests there is a problem of some sort.

     

    I apologize of this has already been answered:

     

    Are you writing the backup to a local drive? Is this a RAID drive?  Is the drive (or drives) ok? Do you have any auto-backup or anti-virus service running? Most Mac users eschew real-time AV software, but it's worth asking.

     

    This sounds like a drive controller or adapter failing back to some tiny RW rate.

     

    Try this: while the backup is running slow, take a "sample" of the Lightroom process from the Activity Monitor.  Take a few samples (say, 3-5) every 2-3 minutes or so. Save these samples and either supply them to Adobe or share them here or both.  I'd like to see what it is actually doing.

     
    |
    Mark as:
  • Currently Being Moderated
    Jan 2, 2011 11:46 AM   in reply to jimtron

    I'm afraid I have no answer, but was wondering if you got one, and found out what the problem was? I'm having a similar issue, long looooong back ups. If you got any good feedback, would really appreciate your sharing my way. Thanks!

     
    |
    Mark as:
  • Currently Being Moderated
    Jan 2, 2011 12:03 PM   in reply to jimtron

    I have only 20,000 images so can't really say how long backup ought to take for 300,000 images.

    But two things to consider: The backup files are quite large, so make sure that there's enough space on the drive where your back-ups are.

    Occasionally I delete older back-ups. The way I work, I see no reason to keep back-ups older than a month because restoring the catalog that much back in time doesn't make sense for me.

    Also, if you are on a Windows machine, defrag the hard drives - where the "working" catalog sits and where the back-up catalog sits - often. Particularly if your catalog is large.

     

    WW

     
    |
    Mark as:
  • Currently Being Moderated
    Apr 5, 2011 11:48 PM   in reply to jimtron

    jimtron wrote:

    . I've got over 250gb of free space on my startup drive (where the catalog is located) and 9gb memory..

     


     

    Any ideas?

    Are you actually backing up to the same drive as where the catalog is located? If so this isn't the best of ideas. It is far better, for obvious reasons, to backup to a separate drive than where your original catalog is located. It will also be faster in most cases.

     
    |
    Mark as:
  • Currently Being Moderated
    Jun 26, 2011 3:07 AM   in reply to jimtron

    jimtron wrote:

     

     

    Oh wait, no I'm not. I do have my LR catalog on my startup drive, but my backups are on a separate, internal drive. Is there any problem with having my catalog on the startup drive?

    Should be no problem are you still having the time consuming issue?

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 2, 2011 7:54 PM   in reply to jimtron

    When you go to back up this window appears:

    Picture 1.png

    and you choose the location with the Choose button.

     
    |
    Mark as:
  • Currently Being Moderated
    Dec 27, 2011 12:44 PM   in reply to jimtron

    jimtron,

    Yes, backing up the catalog is pretty intense for my system too but it should not take hours. My backup takes about 10 min.

    See here what you can do to optimize the performance of LR: http://kb2.adobe.com/cps/400/kb400808.html

     

    With only one drive LR will not work as fast and efficiently as if you had more drives.

    For security reasons it ##########would be better anyways if your catalog backup were on a different drive - even if it was just an USB key.

    WW

     
    |
    Mark as:
  • Currently Being Moderated
    Dec 27, 2011 4:06 PM   in reply to jimtron

    When LR is slow part of the problem is in my opinion that the times for seeking, reading and writing to and from the hard drive add up over time. When you use LR, the hard drive(s) are involved in (a) running the system, (b) running LR, (c) reading from and writing to the LR catalog, (d) reading from and writing to the LR cache, and (e) reading from the image file and - if you have checked this option - write to xmp.

    If all of these tasks are done from one and the same hard drive they can only be executed one after the other to the effect that there will be a queue of tasks for this one hard drive that gets longer and longer and LR slows down. Remember there is only one head on each drive that needs to do all the reading and writing and in between jump to the new location.

    The solution is to spread the tasks over several hard drives, so that the tasks - or at least some of them - can be executed simultaneously.

     

    For these reasons I have the following setup:

    LR "resides" on the C-drive - I think it's called "My Mac" on a Mac; my LR catalog is on the E-drive, while the LR cache is on the D-drive. These three are all internal drives.

    My images are - except for a few - on external hard drives - but one caveat for externals: USB 2.0 is too slow, and the loading of an image from an USB 2.0 external takes forever. Fire Wire (1394a / b) is OK, and eSata is excellent and the best choice for external drives with images. I have no experience with USB 3.0 but I think eSata is faster than USB 3.0.

    In fact speed wise I cannot tell a difference between my eSata external drive and my internal drives - but I haven't done bench tests.

     

    I have also done the following: I opted for rtendering 1:1 previews during import and for never discarding previews. Thus previews load faster in the Develop Module.

    And I have set my cache to a generous 75 GB.

    Does this help?

     

    You also have to remember that with a dual core running at 2.66 GHz your system is at the lower end of systems that can run LR smoothly. So you have to "help" LR with optimal system settings.

    WW

     
    |
    Mark as:
  • Currently Being Moderated
    Dec 27, 2011 4:55 PM   in reply to jimtron

    You wrote:

    "The link you provided on optimizing performance recommends keeping the preview cache with its catalog, which I do."

    The wording chosen in the link is a bit confusing as they call it the previews cache file, when from the context it becomes clear that they task about the previews file - the .lrdata file. This .lrdata file should be together with the catalog (the .lrcat file).

    But the .lrdate file is not the same as the cache of which we can set the size and location when we set the Cache in > Edit >Preferences >File Handling tab.The latter is a temporary cache that LR uses to store previews -  temporarily. And when I suggested to put the Cache on a different drive than the catalog, I meant the cache that is set in >Preferences > File Handling tab. I did not mean the .lradate file.

     

    You say you have 380,000 photos. No wonder your backup takes hours. I have about 25,000 and my .lrcat file is 9.5 GB. So your backup should take at least 15 times longer than mine.

    I also noticed that the more often I do a backup, the less time it takes. Keep in mind that part of the backup process is the optimization of the catalog. In my experience it can take more time than the backup itself.

    WW

     
    |
    Mark as:
  • Currently Being Moderated
    Dec 27, 2011 5:47 PM   in reply to jimtron

    I don't know if it's normal for the backup of a catalog with 380,000 images to take hours, but to me it sounds reasonable under the circumstances and considering how long my backup takes (about 10 min). I think that the backup duration is probably more symptomatic for the speed of your system that it is a problem of LR.

    As far as I know you are the first one who has so many images in the LR catalog - I don't know of anyone else (but that doesn't mean there isn't anybody). Yes, Adobe says that it shouldn't be a problem to have all these images in one catalog. The only caveat I've read from Adobe is that "somewhere between 100,000 and 1,000,000 images your computer will run out of address spaces".But when it's said "it's not a problem" that means that it's possible; it doesn't necessarily mean it'll go fast. And you seem to have no other problems than the long duration of the backup process.

    Have a look how big your catalog file is - it must be in the range of 150 GB. This will take a while for your system to backup.

    WW

     
    |
    Mark as:

More Like This

  • Retrieving data ...

Bookmarked By (0)

Answers + Points = Status

  • 10 points awarded for Correct Answers
  • 5 points awarded for Helpful Answers
  • 10,000+ points
  • 1,001-10,000 points
  • 501-1,000 points
  • 5-500 points