8 Replies Latest reply on Feb 9, 2011 10:33 PM by johnrellis

    View Filter Presets - Yes, they can be iterated, and set, but...

    areohbee Level 5

      Anybody know how to create a view filter?

       

      For example, my first use case would be to filter on PV2003 - anybody know how to do that?

       

      (Back in the good ol' days I would just run a plugin to set custom metadata, then filter on the custom metadata (like Jeffrey Friedl's Focal Length Sort did), but isn't there a better way in SDK3.0? - Even if undocumented, for example, create a view-filter preset file that accesses unexposed fields... - Lightroom would probably have to be restarted to use it though, hmmm..

       

      Any help?

       

      Thanks,

      Rob

        • 1. Re: View Filter Presets - Yes, they can be iterated, and set, but...
          johnrellis Most Valuable Participant

          I have a couple of plugins that get and set view filters using catalog:getCurrentViewFilter() and setViewFilter().  You can reverse engineer the undocumented details (especially for the column browser) by saving filter presets and looking at their definitions in C:\Users\user\AppData\Roaming\Adobe\Lightroom\Filter Presets.

           

          It all seems to work pretty well. Sometimes however, the column browser doesn't appear to update itself properly when you set a view filter with multiple columns and multiple items selected -- but the filter appears to work correctly and show the right photos.

           

          I haven't discovered any mechanism to define new column types (e.g. process verson).  The workaround of adding a plugin metadata field and then filtering on that is slow when you want to filter a large number of photos -- my tests indicate Lightroom can only set plugin metadata at the rate of a hundred or so photos a second.

          • 2. Re: View Filter Presets - Yes, they can be iterated, and set, but...
            JW Stephenson Level 4

            Rob,

             

            I have a couple of shadow fields (folder being one) that I use to filter on like you have described below.  I have a simple plug-in that checks to see if the shadow field is equal to the real field and if not it copies the real field over to the shadow field which can be used for filter bar as well as the smart collections.  I have around 15k images and it takes only about 5 seconds to run this task on all images.

             

            Jeff

            • 3. Re: View Filter Presets - Yes, they can be iterated, and set, but...
              johnrellis Most Valuable Participant
              I have around 15k images and it takes only about 5 seconds to run this task on all images.

              That's much better performance than I've observed, so perhaps you could help me understand how to do better?

               

              Is that 5 seconds to run an incremental update, verifying that the shadow and actual values are identical?  Does it also take 5 seconds to set the initial value of the shadow field, or does that take much longer?

               

              I assume you're calling the batch operations to get the metadata values?  In my tests, batchGetRawMetadata() and batchGetFormattedMetadata() can do about 700 photos/sec on dateTimeOriginal, while you're oberving about 3000 photos/second.  My computer is a little slow (dual-core 1.9 GHz, 4 GB memory, 7200 RPM disk, Windows 7 32-bit), but that shouldn't account for a 4x difference.  Perhaps batchGetRawMetadata ("path") is reading from LR's in-memory data structures, while batchGetRawMetadata ("dateTimeOriginal") is doing a SQL query to the database?

              • 4. Re: View Filter Presets - Yes, they can be iterated, and set, but...
                JW Stephenson Level 4

                John,

                 

                I do have a few more cores but I think the big difference is LR3 vs LR2.  I am still in 2.5 awaiting some performance fixes before switching to v3.  I continue to test LR3 on both my desktop (8 cores) and laptop (2 cores).  Here is what I am finding:

                 

                In version 2.5, it takes approximately 4 times longer to fill the shadow field if all are empty than simply checking.  So my rate of 2-3,000 per second is primarily checking - but that is what it will be doing most of the time once the initial fill is complete.  I see very little difference between my laptop and the desktop for this test.

                 

                Running v 3.0 it much, much longer to fill the fields than in v 2.5 - maybe a slow as 25/second.  Again, I see very little difference between the laptop and the desktop.  The worst part is that in 3.0 the incremental fill takes just as long as the initial fill - this might be related to the issue whereby issuing a Ctrl-S is not longer incremental but updates all images (hopefully an error on Adobe's part that will be corrected soon).

                 

                Conclusion, I think what you are seeing is primarily a difference between v2 and v3 of Lightroom; however, you seem to be getting much of that performance back by using the batch commands which, as a side note, I have not yet implemented since those commands just came out with v3.  I just loop through the catalog and read metadata one image at a time.  The relevant code (not pretty so don't laugh) is basically:

                 

                --###################### task code  ###########################################

                LrTasks.startAsyncTask( function()



                -- get a reference to all the photos in the entire catalog

                local cat_photos = nil

                catalog:withReadAccessDo( function()

                     cat_photos = catalog.allPhotos

                end )


                -- determine number of photos to be processed

                local ncat_photos = #cat_photos



                -- confirm the action since it can be massive on large number of photos

                local message = "Are you sure you want to create the duplicate field for the folder name and set the Metadata errors field of the " .. ncat_photos .. " selected images?"

                local returnbutton = di.confirm( message,nil,"Yes")

                 

                if returnbutton == "cancel" then

                     progressScope:done()

                     return

                end


                 

                -- get access to the current catalog

                catalog:withWriteAccessDo("Construct Captions", function()


                --di.message("Jeff Dialog",tostring(#cat_photos),"info")

                local cap_string = "Checking/Updating " .. tostring(ncat_photos) .. " images "

                progressScope:setCaption(cap_string)


                -- loop through each of the photos selected

                local indexnum = 0

                for i, photo in ipairs(cat_photos) do

                 

                indexnum = indexnum + 1

                 

                local fold = trim(photo:getFormattedMetadata('folderName'))

                local kws       = photo:getFormattedMetadata('keywordTags')  -- retrieves keywordes ENTERED

                local sfold     = photo:getPropertyForPlugin(_PLUGIN, 'shadowFolder' )


                -- remove blanks that don't work for filter bar purposes

                local shadfold  = string.gsub(fold, " ", "")

                shadfold  = string.gsub(shadfold, "_", "")

                local firstchar = string.find(shadfold,"%a") -- finds first alpha character

                if firstchar == nil then firstchar = #shadfold+1 end

                shadfold = string.sub(shadfold,1,firstchar-1) .. "|" .. string.sub(shadfold,firstchar,#shadfold)

                -- if in year/mo format, then insert dash between year and month

                if firstchar == 7 then

                     shadfold = string.sub(shadfold,1,4) .. "-" .. string.sub(shadfold,5,#shadfold)

                end


                -- COPY folder without blanks or _ to duplicate field

                if shadfold ~= sfold then

                     photo:setPropertyForPlugin(_PLUGIN, 'shadowFolder', shadfold )

                end



                end      -- of looping through selected photos


                end )    -- of catalog:withWriteAccessDo


                end)     -- of task

                --###################### end of task code  ###########################################

                Here's to hoping 3.1 brings our performance back!
                Jeff

                • 5. Re: View Filter Presets - Yes, they can be iterated, and set, but...
                  areohbee Level 5

                  Hi Jeff,

                   

                  JW Stephenson wrote:

                   

                  I have a simple plug-in that checks to see if the shadow field is equal to the real field...

                  I think I have that plugin too .

                   

                   

                  JW Stephenson wrote:

                   

                  I have around 15k images and it takes only about 5 seconds to run this task on all images.

                   

                  This is encouraging. Doing about the same thing with my plugin I get results closer to the 25-50 / second in Lr3 (haven't timed it scientifically nor tried in Lr2).

                   

                  PS - If 3.1 doesn't solve some of these performance problems I fear there will be a cyber-riot in the forum...

                   

                  Rob

                  • 6. Re: View Filter Presets - Yes, they can be iterated, and set, but...
                    areohbee Level 5

                    johnrellis wrote:

                     

                    The workaround of adding a plugin metadata field and then filtering on that is slow when you want to filter a large number of photos -- my tests indicate Lightroom can only set plugin metadata at the rate of a hundred or so photos a second.

                    I get about 1/4 - 1/2 half that using the non-batch versions.

                     

                    If I could get the faster speeds of Jeff, the "workaround" solution would be far more palatable.

                     

                    Thanks John,

                    Rob

                    • 7. Re: View Filter Presets - Yes, they can be iterated, and set, but...
                      johnrellis Most Valuable Participant

                      Jeff, thanks for the details -- all makes sense.  I too am hoping that Adobe will invest in the performance of the SDK APIs.  Even doing simple things like enumerating all the keywords takes two orders of magnitude too long.

                      • 8. Re: View Filter Presets - Yes, they can be iterated, and set, but...
                        johnrellis Most Valuable Participant

                        See this post for the speed of catalog:getRawMetadata() in LR 3.3:

                         

                        http://forums.adobe.com/thread/789680