18 Replies Latest reply on Apr 17, 2016 12:28 PM by paulp2355277

    I can never make much space in my hard drive after back up

    paulp2355277

      I regularly back up to a hard drive that's attached to Lightroom.  What I've noticed is the amount of space I have left is always decreasing.  After back up I used to have over 200GB of a 768GB MBP retina available, now I rarely have over 100GB.  At some point I am going to have no space left at all.  I'm not sure I'm doing things correctly.  I have noticed that there are duplicate files when I export the .DNG from my 645Z  or .ARW from my A7RII.  So I have the raw file in LR, the ARW and the .xmp that have been exported (I always name them).  My habit of late to create more space on my hard drive is to find .dng and .arw files in finder, back them up to an external hard drive, and then delete them from my MBP.  I don't touch any of the other un-named files.  What I'm finding though is that even with the hard drive attached, some files are shown as image missing.  I can remedy this with a Time Machine recovery, as I did a time machine back up before I backed up (something I always do) lightroom images. 

       

      Could anyone tell me what I'm doing wrong?  What's the best way to ensure I have plenty of space?  I can't just buy new laptops all the time when I run out of space.  I save most of my finished images from PSCC to highest quality jpg, so the finished images aren't taking up that much space overall, but these aren't archived on LR as I prefer to use the tag system that Mac uses.

        • 1. Re: I can never make much space in my hard drive after back up
          Jao vdL Adobe Community Professional & MVP

          The best approach I found is to always import in date segregated folders to your internal. Make sure these get backed up by time machine or some other backup mechanism and then if you are not actively using the images in a dated folder anymore copy it whole to an external hard disk where you maintain the same year and date segregated structure. Then in Lightroom it is an extremely simple case of right (or control) clicking the folder you moved in the library side bar and selecting "update folder location" and pointing it to the new location for the folder on the external disk. All your images should be accessible as long as the external is attached. Now you can delete the folder from your internal hard disk and regain the space (don't forget to empty the trash).

           

          My main machine is a retina with a 256 GB disk and I easily maintain a 100's of 1000 image Lightroom catalog on it in addition to Photoshop and a lot of other CC software I use (premiere mostly). Most of my images and video lives on a big RAID array, but my several months of most recent images lives on the internal. You should easily be able to work with a 768 GB internal. Are you deleting old Lightroom catalog backups? By default Lightroom creates the backups of the catalogs it makes every week or so right next to the catalog file in a "Backups" folder. You don't need to keep a lot of these backups around. I usually only keep 2 or 3.

          • 2. Re: I can never make much space in my hard drive after back up
            Just Shoot Me Adobe Community Professional

            By default all MBP models use a system to make totally useless backups of the system when a Time Machine drive is not connected to the computer. They are called Local Snapshots. To read more about it look at this page.

             

             

            Time Machine - Frequently Asked Questions 30. What are Local Snapshots?

            • 3. Re: I can never make much space in my hard drive after back up
              trshaner Adobe Community Professional & MVP

              paulp2355277 wrote:

              My habit of late to create more space on my hard drive is to find .dng and .arw files in finder, back them up to an external hard drive, and then delete them from my MBP. I don't touch any of the other un-named files. What I'm finding though is that even with the hard drive attached, some files are shown as image missing.

              The LR Catalog file contains location data for every file and folder you've Imported into LR. If you Move, Delete, or Rename a file or folder from outside LR using Finder it will show a '!' Missing icon on the thumbnails. You should only perform these operations from inside LR using its tools for these purposes.

              Re-linking Files and Folders in Lightroom « Julieanne Kost's Blog

               

              It sounds like you are running out of space on your system drive and will need to move some or all of the picture files to an external drive. LR creates Preview image files that can take up a considerable amount of drive space, which is another reason why you're seeing Free Space dropping quicker than expected. For performance reason it's best to keep the LR Catalog file and Previews folder on the local system drive.

              • 4. Re: I can never make much space in my hard drive after back up
                paulp2355277 Level 1

                I have been moving the files  to externals within LR for some time.  But the amount of hard drive space I have left drops too quickly in spite of this.  From what I can see, when I export, LR creates another image as well as the raw folder - I have done this with DNG files from my 645Z and can still access the files from within LR with the hard drive attached.  I have backed up these files to my main external hard drive.  There is going to be a time not far from now when I simply don't have any space on my hard drive at all inside of backing up by moving files within LR.  I'm not very technically minded, so there's likely to be something that I'm doing wrong, but all my completed edits in jpg total only 55GB with a further 30GB on an external drive, and yet I have lost over 100GB in space somewhere and I can't fathom out why.  Could be a mac issue unrelated to images  of course, but images are most of what I use my MBP for...

                • 5. Re: I can never make much space in my hard drive after back up
                  Just Shoot Me Adobe Community Professional

                  Go into the Mac preferences and turn Time Machine Off. See if that frees up a lot of space. You can turn it back on if you like.

                  • 6. Re: I can never make much space in my hard drive after back up
                    Jao vdL Adobe Community Professional & MVP

                    By default all MBP models use a system to make totally useless backups of the system when a Time Machine drive is not connected to the computer. They are called Local Snapshots. To read more about it look at this page.

                    These local snapshots are actually incredibly useful to save you from accidentally deleted file issues especially in cases where you are traveling away from your backup drive. I've recovered previous versions of documents that I saved over using this. These local snapshots take very little space as they only save changed files since the most recent external backup and automatically size themselves if you need more space. It automatically deletes them if your drive gets below 20%. So these are nothing to worry about. You shouldn't rely on them but they can be lifesavers.

                    • 7. Re: I can never make much space in my hard drive after back up
                      paulp2355277 Level 1

                      I don't keep my Time Machine attached to my laptop and have switched time machine off.  I still only have 124GB remaining whereas a month or two ago I was getting 180 - 190GB after moving files to hard drive.

                      • 8. Re: I can never make much space in my hard drive after back up
                        Jao vdL Adobe Community Professional & MVP

                        From what I can see, when I export, LR creates another image as well as the raw folder - I have done this with DNG files from my 645Z and can still access the files from within LR with the hard drive attached.

                        Are you keeping all your exported files? You only need the copy inside your catalog which can reside on your internal or your external. When exporting from Lightroom you just generate temporary copies for use somewhere else. When you're done with them, just delete the exported copies. You can always quickly recreate them if you need them again.

                         

                        all my completed edits in jpg total only 55GB with a further 30GB on an external drive, and yet I have lost over 100GB in space somewhere and I can't fathom out why. Could be a mac issue unrelated to images of course, but images are most of what I use my MBP for...

                        Check your previews folder (right next to the catalog file). How large is it (do a get info on it)? What preview size are you creating by default (check your catalog settings inside Lightroom). You generally do not need 1:1 of everything. Do you create any smart previews? Check the size of the smart previews folder if you do. The previews and the smart previews can easily take 100's of GB of space if you are not careful. How many backups of the catalog are you keeping? Delete the old ones.

                        1 person found this helpful
                        • 9. Re: I can never make much space in my hard drive after back up
                          Jao vdL Adobe Community Professional & MVP

                          The time machine backups CANNOT be the source of your problem. You just need to figure out what is taking up so much space. It is likely backups of the catalog and the size of the previews or smart previews.

                          1 person found this helpful
                          • 10. Re: I can never make much space in my hard drive after back up
                            Just Shoot Me Adobe Community Professional

                            You and I differ completely on this topic. They are not useful in any way unless you are the type of user that Willy Nilly Deletes files without knowing what those files are and what they are used for. In any event any file the user deletes is stored in the Trash Bin and can be recovered.

                             

                            If by chance you are the type of user that goes into the hard drive and starts deleting file for no good reason then you should suffer the consequences. Then maybe you will learn not to do that. If not then you will learn how to reinstall the OS and your programs.

                             

                            Again if you actually know how to use a computer you make incremental backups of files you are changing to a different file name, like with a Date and Time stamp.

                            They are nothing to worry about if you have lots of disk space. There are many reports of that system not working which will cause you to run out of disk space.

                            Then why is it they are only activated on mobile Mac's and not on desktop Mac's?

                             

                            A lifesaver is learning how to actually use a computer.

                            • 11. Re: I can never make much space in my hard drive after back up
                              paulp2355277 Level 1

                              I deleted one of the ARW files, keeping the .xmp file, and after doing so, the file became missing in LR.  I chose one that wasn't a keeper so less of a risk.  When I have deleted exported files that .dng before, I still have access to these files when I attach the hard drive.  There are over 100GB of .ARW files most of them un-named as I haven't exported them, but inspite of moving everything they still show in my hard drive.  I don't think I have duplicates there, though for some reason, not all files were moved.  The only difficulty is if I have files from my old A7R there and they have the same extension and therefore it sees the files I am moving as duplicates.  But we would be talking a lot of images here...

                              • 12. Re: I can never make much space in my hard drive after back up
                                paulp2355277 Level 1

                                I'm no computer expert (nor am I complete newbie), and I don't think it's a Time machine issue either as it is only connected when I move files from my MBP to an external drive.  I stopped having a time machine switched on and attached before because of the extra back up that was always running. 

                                • 13. Re: I can never make much space in my hard drive after back up
                                  paulp2355277 Level 1

                                  It seems that it may well be a duplicates issue.  Going through older A7RII files, they are still in the catalogue and photo showing even though I moved them.  I tried to move a single image, and it said it was already there.  So it could be that on a previous back up I have an A7R image and it won't back up - unfortunately Sony don't allow you to rename files.  Is there a way around this apart from getting a new external hard drive and moving to that?

                                   

                                  Also I should double back up my hard drives.  Is there a way of doing this within LR?

                                  • 14. Re: I can never make much space in my hard drive after back up
                                    Jao vdL Adobe Community Professional & MVP

                                    paulp2355277 wrote:

                                     

                                    I deleted one of the ARW files, keeping the .xmp file, and after doing so, the file became missing in LR. I chose one that wasn't a keeper so less of a risk. When I have deleted exported files that .dng before, I still have access to these files when I attach the hard drive. There are over 100GB of .ARW files most of them un-named as I haven't exported them, but inspite of moving everything they still show in my hard drive. I don't think I have duplicates there, though for some reason, not all files were moved. The only difficulty is if I have files from my old A7R there and they have the same extension and therefore it sees the files I am moving as duplicates. But we would be talking a lot of images here...

                                    Can you take a moment to describe exactly what you are doing? I don't think I understand your workflow at all. Are you exporting all your files to dng and then deleting the originals? You should not do that! When you want to move files off to an external. Best thing is to move the originals folder by folder and reconnect the folders. You can also do the move inside Lightroom by just dragging the folder to the external hard disk in the side bar of Lightroom. In previous versions of Lightroom this was a bit dangerous so I generally advise against it but it works just fine in general.

                                    • 15. Re: I can never make much space in my hard drive after back up
                                      paulp2355277 Level 1

                                      I learned not to do anything outside LR a number of years ago.  I move everything across by selecting all images and then moving them across to the hard drive.  That bit was done already.  After that, I go to finder and search for named files either .dng or .arw and copy these to the hard drive - these are exported files.  I leave .xmp files there.  After the exported files are copied, I disconnect the hard drive and delete the .dng files moving them to the trash.  Only named ones, not numbered, as I always name my exports.

                                       

                                      Obviously in moving files in LR, it won't move duplicates, and I think this is my issue as some A7R files will have the same file names as A7RII.  It would be great if there were a way around this.

                                       

                                      To be clear, I'm only exporting files that I have done basic edits on and then working further on them in PS.  The only files I have actually deleted are some of the exported .dng files.  Not the original ones, but the named ones.  These are copied to a hard drive before I delete them.

                                      • 16. Re: I can never make much space in my hard drive after back up
                                        Jao vdL Adobe Community Professional & MVP

                                        Ah. So you move your originals to the external inside Lightroom? Then you also have exported dngs and arw files that you have somewhere on your internal hard drive? That seems like unnecessary duplication to me. You copy these over (I would make sure to copy the xmp files with the are files btw as they are only meaningful together with the raw file) manually to your external disk.

                                         

                                        After the exported files are copied, I disconnect the hard drive and delete the .dng files moving them to the trash. Only named ones, not numbered, as I always name my exports.

                                        Just stating the obvious perhaps but realize that the hard drive space is only reclaimed when you empty the trash.

                                         

                                        Obviously in moving files in LR, it won't move duplicates, and I think this is my issue as some A7R files will have the same file names as A7RII. It would be great if there were a way around this.

                                        I am not sure that I understand this. Are you moving the raw files inside Lightroom one-by-one into a giant folder on your external and are you getting naming conflicts? You should not use a single folder to store all your images. This is a recipe for problems such as this. Simplest solution as I stated above is to have folders segregated by date and move these folders instead of the images individually. Another important thing is to make sure your camera is set up to number images sequentially and to not restart numbering every time you reformat your cards. Many cameras are setup by default to start at 0001 every time you insert an empty card which can easily lead to numbering and filename conflicts. Of course even with sequential numbering you can get conflicts as soon as you have more than one camera or when you have more than 10k images in a folder which is one small reason why segregating by date is a good idea.

                                         

                                        To be clear, I'm only exporting files that I have done basic edits on and then working further on them in PS. The only files I have actually deleted are some of the exported .dng files. Not the original ones, but the named ones. These are copied to a hard drive before I delete them.

                                        So you export a file to dng and then open that from the finder in Photoshop? That is not necessary. You can simply do "Edit in Photoshop" from Lightroom itself and it will open directly in Photoshop. Saving from Photoshop will generate a tiff or psd file that will be placed right next to your original raw file and will automatically show up in Lightroom. Then it is easy to move the photoshop edited file to your external by just moving the folder again in Lightroom and it will stay near your original raw file.

                                        1 person found this helpful
                                        • 17. Re: I can never make much space in my hard drive after back up
                                          paulp2355277 Level 1

                                          I've never used the edit in photoshop.  I've always exported selecting the option in the export dialogue to open in PSCC. Will definitely try that out, but I usually save to jpg unless there are likely to be banding issues in doing so.

                                          • 18. Re: I can never make much space in my hard drive after back up
                                            paulp2355277 Level 1

                                            Setting up an individual folder for the A7RII files seems to be doing the trick for creating space.  Thanks for the help...