Hi John/Don -
I appreciate your thread on duplicate files....I'm running int othis roblem majorly too. How I get into the mess int he first place is not a consistent mess up, but it happens....and here's how Lightroom falls on it's face....it's a crazy bug almost.
Suppose I have a group of DNG's or NEF's etc.... located in a subfolder for the project. I have a workflow that created a structured sub-directory stack that is exactly the same for every project....the sub's are differentiated by ranking best to least. It is very common that I move files between these folders in the ranking and editing process. When I use LR2 to drag files from one directory to the other, if there is a single duplicate file name, the entire copy is terminated - WHAT WERE THEY THINKING?
I cannot believe that LR2 does not offer the option to SKIP, Overwrite, rtc.... any dupes This is the same workflow and an export to a target folder provides...so the technology is here.
The only way I have figured out how to do it - and it manual, is to show the source folder in the finder and use the finder to reconcile any duplicates on a drag and drop, telling it to overwrite them or not do so...that works, but then heading back into LR, it is then missing the files and more steps are required there....
So you guys think third party computer wide duplicate file finders are a better method? What app do you like for Mac?
Darrin Harris Frisby
http://www.10PointProductions.com/
I think u should use an application to delete duplicate files, i know the name of that software which can duplicate files delete and folder called duplicate finder 2009.
Try to use app which is designated for finding and removing duplicate images - Duplicate Checker.
Interesting recommendation from a first time poster for an application that's PC only.
For Macs, try http://www.araxis.com- "Find Duplicate Files".
Here is a tool to find and remove duplicates with an accuracy in a minute, Duplicate remover helped me to find and delete duplicate files from the pc, it has a great features like instant preview that allow you to select tha file before deleting them or move.
Suppose I have a group of DNG's or NEF's etc.... located in a subfolder for the project. I have a workflow that created a structured sub-directory stack that is exactly the same for every project....the sub's are differentiated by ranking best to least. It is very common that I move files between these folders in the ranking and editing process. When I use LR2 to drag files from one directory to the other, if there is a single duplicate file name, the entire copy is terminated - WHAT WERE THEY THINKING?
They were thinking that they gave you tools that make things easier for you than to use folders as your organizing methodology. They are deliberately trying to make people move away from organizing via folders. They have designed it so that your workflow can be streamlined and made more efficient by using Lightroom tools instead of folders.
The only way I have figured out how to do it - and it manual, is to show the source folder in the finder and use the finder to reconcile any duplicates on a drag and drop, telling it to overwrite them or not do so...that works, but then heading back into LR, it is then missing the files and more steps are required there...
You cuold use Lightroom tools, such as stars, colors and pick/delete flags, plus keywords, collections, metadata to accomplish everything you are trying to do, without moving a photo from folder to folder. Works for me. If you absolutely have to continue using your folder structure and existing workflow, rather than streamlining your workflow with Lightroom features, then it seems that this is a type of workflow where Lightroom will frustrate you. So I recommend you consider changing your workflow so that you use Lightroom tools instead of folders.
Paige & All - Thank you for you for your replies...I am on Max OSX and so the PC programs wont work, but I'll try the abraxis program.
Paige, You sound committed to yoru ressponse and I too am thoughtful about my workflow and took yoru words to heart adn thought about what is an absolute in what I do and why. I thought I'd put it out there to see what kind fo hit you had adn wher I may be able to lighten my load.
First, I do heavily rely on OS Folders versus allowing Adobes metadata methods define the distinctions within my body of work. Here' a few reasons why and what I have done to solve my own issues
1 - As a professional photographer, I maintain a live database of more than 3 TERRABYTES of original content on my file server. This body of content is FAR TOO LARGE for any single Lightroom database to handle. In fact, at the file level, we are talking about 250,000+ DNG's or NEF raw files and their respective children in TIFF or JPEGS, organized, project by project. By result, I deviate imediatly from a fundamental premise of Adobe's workflow - 1 database. I DO NOT MAINTAIN A SINGLE DATABASE FOR MY IMAGES, BUT RATHER BUILD A BRAND NEW DATABSE FOR EVERY SINGLE PROJECT I SHOOT AND IT LIVES WITHIN THE FOLDER FOR THAT PROJECT ADN IT'S FILES. Reason number ONE for folders
2 - FILE COUNT - In my experience Lightroom chokes when too many files are put into a single database...somewhere between 50,000 and 80,000 brings the application to a useable halt....I operate on a Mac with 8 cores running at 2.5+ GHZ and have 32 GIG of memory....so this machine is not the bottle neck. As such, I have a fodler for every project / shoot. I then create a brand new LR catalog for that shoot and house it within the project folder. Simple as that. Reason number two for not using adobes rankings, colors adn flags for marking.
All of this said, NO DOUBT, I use colors, rankings and flags like crazy while managing my files from a shoot. The 16,000 images I took at Burning man have required everytrick int he book to manage.
HERE'S MY MAIN ISSUE WITH LR AND YOUR POSITION AND RESPONSE
I love Adobe and frankly use most of their software to create and manage my workflow and images. That said, it is NEVER WISE, to rely on a single companies methodologies and thinking to structure your own work. Here is a perfect case where their minds do not align with mine and I cannot achieve what I need....and frankly, their minds do not offer a solution to my issue.
But your know who has figured it out and addressed my problem perfectly...? Apple (and I am sure Microsoft too)...how did they do it? At the basic OS level, when moving files from one folder to another, the OS recognizes duplicate file names and immediately checks date created and bam...it asks the SINGLE MOST IMPORTANT QUESTION that I wish Lightroom would ask: "AN OLDER (OR NEWER) FILE WITH THE SAME NAME ALREADY EXISTS IN THIS LOCATION. WOUDL YOU LIKE TO REPLACE THE FILE WITH THE ONE YOUR MOVING?"
Lightroom has this functionality in the EXPORT dialouge workflow but does not in the simple drag adn drop move features....GOD I WISH IT DID.
ADOBE...if your listening out there....call on me to be more clear if I am not.
Darrin Harris Frisby
10pointproductions.com
jacktors wrote:
Paige & All - Thank you for you for your replies...I am on Max OSX and so the PC programs wont work, but I'll try the abraxis program.
Paige, You sound committed to yoru ressponse and I too am thoughtful about my workflow and took yoru words to heart adn thought about what is an absolute in what I do and why. I thought I'd put it out there to see what kind fo hit you had adn wher I may be able to lighten my load.
First, I do heavily rely on OS Folders versus allowing Adobes metadata methods define the distinctions within my body of work. Here' a few reasons why and what I have done to solve my own issues
1 - As a professional photographer, I maintain a live database of more than 3 TERRABYTES of original content on my file server. This body of content is FAR TOO LARGE for any single Lightroom database to handle. In fact, at the file level, we are talking about 250,000+ DNG's or NEF raw files and their respective children in TIFF or JPEGS, organized, project by project. By result, I deviate imediatly from a fundamental premise of Adobe's workflow - 1 database. I DO NOT MAINTAIN A SINGLE DATABASE FOR MY IMAGES, BUT RATHER BUILD A BRAND NEW DATABSE FOR EVERY SINGLE PROJECT I SHOOT AND IT LIVES WITHIN THE FOLDER FOR THAT PROJECT ADN IT'S FILES. Reason number ONE for folders
You most certainly are dealing with a vast number of images. And you are right, Lightroom's Catalog (its database) is unlikely to efficiently grow to such size. But that might not force you to manipulate your images from folder to folder so much. Maybe this would work for you:
As you work with the images from the shoot, flag, label and rate them as you wish. That will let you filter your Folder view by whatever criteria is important at the moment - showing all your 5 star images, for example. You could even use Smart Collections to make this more automatic. For example, create a Smart Collection for the shoot that finds and displays all your 5 star images. Every time you rate an image with 5 stars in your shoot Folder, it will automatically be placed in the 5 star Collection. You could make as many Smart Collections as need to track things such as client approvals (a Smart Collection for the keyword "client approved"), adjustment progress (a Smart Collection for images with a yellow tag if yellow meant adjustments completed), and so on.
2 - FILE COUNT - In my experience Lightroom chokes when too many files are put into a single database...somewhere between 50,000 and 80,000 brings the application to a useable halt....I operate on a Mac with 8 cores running at 2.5+ GHZ and have 32 GIG of memory....so this machine is not the bottle neck. As such, I have a fodler for every project / shoot. I then create a brand new LR catalog for that shoot and house it within the project folder. Simple as that. Reason number two for not using adobes rankings, colors adn flags for marking.
The approach outlined above will keep the Catalog (database) size in check. It will also let you place shoot folders on different hard drives as you fill them up.
I love Adobe and frankly use most of their software to create and manage my workflow and images. That said, it is NEVER WISE, to rely on a single companies methodologies and thinking to structure your own work. Here is a perfect case where their minds do not align with mine and I cannot achieve what I need....and frankly, their minds do not offer a solution to my issue.
If its important to you not to use Collections, and to have your images organized only by the file system, then you may find using Adobe Bridge and Photoshop more in line with your thinking.
Bob
Try a tool from Ashisoft to remove duplicate files from the drives.
Download Free: http://www.ashisoft.com
aaiight don!
I have read your problem about duplicate files , I also faced the same problem few days back.and i used one that was
absoludicrous
.Though I manage my duplicates using some other tool , well its advanced system optmizer . just try iot once as it comes bundled with other applications as well.
Its advanced system optmizer . use its duplicate files remover.It manages all the duplicates found on ur system. just select the drive or drives from u need to remove duplicates.chose ur extensions or file types.
just download it from http://download.cnet.com/Advanced-System-Optimizer/3000-2094_4-1014765 9.html?tag=mncol this link..
Use its & experience its wonderful results
..
Cheers !!
I ran into a problem of duplicate files and someone told me about http://www.dublicateFilesDeleter.com
well... to solve your serious duplicate files problem... my advice is to use an automatic solution that will search for unnecessary files in your disk.. one I know is this http://www.dublicateFilesDeleter.com
That is a good program I tried http://www.dublicatefilesdeleter.com/
I ran into a problem of duplicate files and someone told me about http://www.duplicates-finder.com
Let me add my 5 cents here.
The best duplicate file finders that I have used are:
Duplicate File Finder-finds all file duplicates.
Duplicate MP3 Finder- can find similar songs
Duplicate Image Finder - can find similar images
You can check also this duplicate file finder web site.
North America
Europe, Middle East and Africa
Asia Pacific