11 Replies Latest reply on Aug 2, 2015 11:13 AM by prs12342

    Face recognition disappointment

    nosparetime

      Let's talk about the face recognition feature. It's very disappointing.

       

      First, an example. This is Angus. Angus has a well selected set with some variation.

      HeIsAngus.png

      Lightroom thinks these people are Angus. This is not the "Find Similar" set. This is the section of the "Unnamed People" list that had Angus as the best match.

      WeAreALLAngus.png

      None of the suggestions are useful, or even seem relevant to the initial face. This is just one example of a huge number.

       

      The user interface is its next failing. It requires a quirky mix of keyboard and mouse operations. Selecting names isn't from a list of suggestions until a key is typed. After choosing, the selection is editable/being edited and doesn't get assigned until focus is lost. The suggestions aren't by quality, just text matching. The performance is extremely poor with many images in the filmstrip.

       

      I was expecting a high quality UI, especially considering the nice interface Map and Book had when they were rolled out.

       

      To be useful, the feature needs:

      1. A method to adjust the sensitivity of the face detection (many missed faces when oriented in profile)
      2. A method to rescan images for new face regions
      3. A method to adjust the sensitivity or match quality before making a suggestion
      4. A method to sort the suggestions by confidence
      5. A method to group people (hierarchically or otherwise): e.g. by family, friends, team, school, grade -- think Collections, but for people
      6. A method to mark a face as "Unknown"
      7. A method to review Unknown faces or reclassify at a later date
      8. Automatic grouping of Unknown faces by similarity
      9. A method to set a valid date range for a person (i.e. birth / death... since his death, my grandfather has apparently attended numerous weddings and team sports events)
      10. A method to mark a suggested person as "Not present" (at this event/shoot/date)
      11. (algorithm) If a series of shots is detected (burst mode), take advantage of that information (many ways to use this)
      12. (algorithm) If people have been identified in an image together, increase the chance they might be together in future images (think team pictures, wedding photos, etc.)
      13. A better, dedicated interface (make it a module). Include Speed ID mode:
        1. Show face to classify
        2. Show grouped faces that will also use this classification
        3. On right, show top 5 candidate faces, caption with names.. sixth "blank" face with text entry box for creating new entry
        4. Click to select, or use 1-5 hotkey
        5. hotkeys (Y)es if match is correct, (N) if match is incorrect, (U) unknown person, (S) skip, (X) not present / exclude
        6. Auto-advance if capslock is on

       

      Please sort this out.  It sucks.

       

      (In 2011, Adobe Elements face recognition was much better than this. Picasa and iPhoto were better in 2009!)

       

      If anyone has any suggestions, or has found ways to make this feature usable with a library of >3000 images. Please let me know.

        • 1. Re: Face recognition disappointment
          genefama Level 1

          Yours are all true criticisms and good suggestions.


          The weird mouse-keyboard interface needs to be fixed pronto. If you use a Wacom tablet it's almost impossible to change the name in a field without LR canceling the operation. Also, once someone in a picture has been identified, isn't it a low probability they could also be their brother in the same photo, in spite of the vague family resemblance? And why doesn't it take the timeline into rough account? This might be hard to write as an algorithm but if someone was an infant in 2013, shouldn't Lightroom know enough not to suggest the same person as an 80 year old in 2008? Plus can't it tell an illustration on the cover of a comicbook or record album from a real face? And it's computationally very slow, lots of spinning wheels and clicks taking place 30 seconds after you make them. Is this why they tell you to wait till its done running before you do anything with the results? (They suggest this in the instructions but don't say why, which I can't stand. Tell us why!)


          MY wife got upset because she passed by my monitor while Lightroom was asking if an Emperor Palpatine PEZ dispenser was her—and she's an attractive woman!

           

          The face detect functionality in Lightroom is so uncharacteristic in terms of how it's thought through and executed that it deserves a complete scrapping and re-doing. It's just not up to the high standards of the program.

          • 2. Re: Face recognition disappointment
            genefama Level 1

            It'd be good if rejecting a face also rejected other instances of that face and if deleting a face deleted all instances of the face (after putting it in a holding pen for review). Right now I think you have to reject and delete them individually until every shot of a person you don't know is purged.

             

            Also the spraypaint tool should be available when face-tagging. (You're tagging, right? Why aren't all the tagging tools at your disposal? Not to be snarky, but did this really not come up in any of Adobe's development meetings?) You should be able to scroll through a lot of images and tag the same person in the random places he or she appears, especially since increasing the number of confirmed faces for any given person doesn't seem to narrow the range of faces Lightroom continues to wrongly suggest.

            • 3. Re: Face recognition disappointment
              nosparetime Level 1

              It doesn't seem to reassess a proposed / unconfirmed match (perhaps it does but it's slow?).

               

              If I add someone to the database that wasn't there before, it needs to recheck the unconfirmed matches (at least within a certain time range of the update). This might make the process less tedious than it is now.

              • 4. Re: Face recognition disappointment
                genefama Level 1

                I find that the "Confirmed/Similar" view where you select an individual person and then look at suggestions for that person works better than the big view where it shows everyone in alphabetical order. At least in the individual view the first few rows of suggestions or the first few thumbnails (depending on how many photos you have of the person) has a chance of being somewhat accurate, especially if they were shot in continuous mode.


                But If they're oriented in any kind of a profile or are wearing sunglasses, they won't show up and you'll have to wait till you come across them as suggestions for other people, many of whom they bear zero resemblance to.

                • 5. Re: Face recognition disappointment
                  _x57_

                  Thank you for summing up these suggestions. I completely agree and i am disappointed about the feature too. I would like to add a suggestion to your list:

                   

                  (14) Show face boxes in the library grid view so that you do not have to cycle through all photos to check if everything was found, this is exactly what i do not want to do anymore.

                   

                  (or is this already possible somehow?)

                  • 6. Re: Face recognition disappointment
                    genefama Level 1

                    I figured out some things that have been working for me and might work for others too if their catalogue and organization is anything like mine. When I upgraded to LR6 I took it up on its offer to find faces in my entire catalogue. In hindsight, this might have been a mistake as it's taken a whole week and I only expect it to finish the task tonight.

                     

                    I've found that the amount of calculating and re-sorting it has to do in any view that includes the entire library is a non-starter. Lots of spinning wheels and clicks occurring a half minute after I make them. Lots of mistaken suggestions too, as discussed earlier in the thread.


                    What has worked much better for me is to go to individual folders one at a time and click on the face detection button. In my case the folders are sorted by year with monthly sub-folders. First I accept all the faces it gets right. Next I drag and drop faces it doesn't get right onto the correct stacks of confirmed people. Dragging thumbnails in this view is reasonably responsive. Clicks typically occur within a second after I make them.


                    My library goes back to 2004, and some months have a few thousand pictures, but there seems to be no way around this confirming process. This subfolder method is only faster because the software responds more quickly when the number of images is lower. The confirmed stacks are also easier to drag shots onto because the amount of identified faces in any given month is a much smaller set than the superset of every identified face in the library. I usually can see them all in the top half of my monitor and simultaneously see several rows of suggestions in the bottom half of the monitor that I can identify for the first time or drag onto the confirmed stacks.


                    I hope this makes sense. I'm still not sure if it's identifying all faces and would love the ability to re-run the face search and have it offer up only new faces, the way the import function allows you to see only photos that are new and not already in the catalogue. That way I could double-check that it didn't miss anyone. But for the time being, this is a quicker and slightly less frustrating way to tag my thousands on thousands of images. Hope it helps you too.

                    • 7. Re: Face recognition disappointment
                      genefama Level 1

                      Ugh, my suspicions are confirmed: The facial detection feature bypasses tons of faces. At least a third and up to half of my photos of people weren't captured in Lightroom's initial search. Drawing bounding boxes around all those faces would be a monumental task. Hopefully future versions will allow you to run multiple face searches until they're all captured.

                       

                      If this feature worked it'd be a huge help but so far it's been an exercise in frustration. I don't recommend anyone use it or upgrade In hopes of using it. Save yourself the aggravation.

                      • 8. Re: Face recognition disappointment
                        BDoug Level 1

                        Yeah, I just discovered how poor the face recognition is. As you say, Picasa was much, much better years ago (last time I used it).

                        • 9. Re: Face recognition disappointment
                          Jim Waltham Level 1

                          Yes, this is super disappointing. I did NOT let LR go through my entire library to search for faces, just selected folder by folder.In addition to the other comments, I find that it's incredibly slow since it looks through an entire folder for faces. Some of my folders have over 1,000 photos. It's selected a number of trees and posts for me, sometimes even guessing the people names of said objects. Since the search through a folder is so slow, I'd like the option to select the photos that have faces and then let LR do its thing with that smaller selection. If 2/3 of a folder is landscapes, why can't I tell it to ignore that whole group of photos?

                          Definitely not ready for prime time, and a serious disappointment that Adobe released such a beta product. I'm into over an hour now just having it search in a single folder, forget about finding anything useful.

                          • 10. Re: Face recognition disappointment
                            Bob Somrak Level 5

                            Agree.  Face Recognition need a MAJOR overhaul along with some improvement in HDR and Pano.  You need to be able to rerun the face recognition on a set of photos and/or folders/collections.  You should also be able to select the photos the initial FR is done on and not just by folders.  Poorly thought out tool as far as I can tell.

                            • 11. Re: Face recognition disappointment
                              prs12342

                              LR feedback.JPG

                              One disappointing aspect of facial recognition that others have not mentioned is that in my case the face the bounding box tends to be in the upper left, whether there is a face there or not. See the attached screenshot. Of the 24 photos in the screenshot only one captured a face (middle row left). In the others, sometimes there are faces but the bounding box does not center on the face. This was scanned early in the release of the facial recognition but even if fixed(?), I find no way to rescan. Also, the second photo asks if it is me (Paul?). What kind of algorithm are they using? There are hundreds of pictures of me and that woman with a microphone in front of her looks nothing like me. Please, Please fix it. As others have mentioned Picasa was better years ago.