1 Reply Latest reply on Dec 2, 2016 6:51 PM by LoweMo.photo

    Many SDK methods yield, preventing robust plugins

    johnrellis Most Valuable Participant

      A number of SDK methods do a yield, allowing other tasks to run while the method is being called.  This makes it extremely difficult (practically impossible) to implement robust plugins.


      The basic advantage of cooperative, non-preemptive tasks is that the programmer doesn't need to worry about locking.  It is trivial (or should be trivial) to write blocks of code that are guaranteed run atomically.  But that guarantee goes out the window when many SDK methods can yield to other tasks.


      See my post in the feedback forum for more details: Lightroom SDK: Many methods yield, preventing robust plugins | Photoshop Family Customer Community

        • 1. Re: Many SDK methods yield, preventing robust plugins
          LoweMo.photo Level 1

          This seems to be a statement rather than a question, but I agree that this could at least be a bit better documented. I ran into the same "issue" very recently when I had to call an "expensive" keyword-list-collecting function that has to start from a call within LrTasks.startAsyncTask()


          After endlessly debugging my tables and getting varied results, I realized that a simple period of LrTasks.sleep()"fixed" things. My solution (as it currently stands) was to assign the return values of the function I was calling to global variables and ran it before some other "heavy work" was started (from another .lua file, hence my use of a global variable) and some time before the plugin would be trying to access the values of the new global variables. Since those globals wouldn't even exist until the return finally happened, I could then just use a while loop, inserted at the point (in another file) where the values of those variables were needed, and wait for them to exist (i.e. not be nil... or time out). It was a simple solution that got around the issue of trying to access a table that didn't yet exist (worse, before, I was trying to access elements in large tables that were still being written to, with wildly random results and lots of errors coming from logic assumptions). There should be a (simpler) way to indicate that one SDK process beginning depends on the completion of a particular previous process.


          To illustrate a bit better, my context was contributing to a Lightroom plugin on Github that uses an export process to send files to Clarifai (a "computer vision" keywording service). Just before the files are exported to the service, I call on a function to collect all keywords (along with their "ancestry paths" in a hierarchical keyword set) into a lookup table. Each "suggestedTagName" can correspond to multiple keyword objects (though that is something a user should try to avoid, where possible, by cleaning up their keyword list):


              local catalog = import 'LrApplication'.activeCatalog();
              _G.AllKeys, _G.AllKeyPaths = require 'KwUtils'.getAllKeywords(catalog)

          When it comes time to build the tagging dialog with the results of the request, we want to ideally add existing keywords to photos. Assuming you have a controlled vocabulary (with a hierarchical structure), you need to find the keywords by the names returned. Iterating the keywords to find all keywords by a given name and create a checkbox for each (on a larger group of photos) was too "expensive", so this function collects all keywords into the table (indexed by keyword names). I won't go into further details (there are many) here, but this is the way I currently pause operations to be sure the keywords table is ready for access (from the lua file that generates the tagging dialog):


              -- Before we trim down our lookup table for keywords and keyword paths, we should
              -- be sure the process of populating our global variables for these has completed.
              local sleepTimer = 0;
              local timeout = 30; -- If it doesn't complete within 30s more, something is wrong. 
              while ((_G.AllKeys == nil) and (sleepTimer < timeout)) do
                  sleepTimer = sleepTimer + 1;


          Now that I think about it a bit more, I'll probably move both of those blocks into my KwUtils.lua file (helper functions for keyword-related tasks), the first block into my "getter" function and the second into a waitForVariable helper function (probably in the KwUtils-dependent LUTILS.lua set of utility functions that I found myself inserting into every project in a more haphazard way). My intention is to share my set of utility functions (inspired by Jeffrey Friedl's JSON.lua package, which the aforementioned project also makes use of) via CC attribution license. I've included them in a few of my own projects, now, and they really help simplify some things.


          Okay, I've just re-written that second block into a helper function, so all we have to do, now is call:

              local timeout = 30;
              local timeWaited = LUTILS.waitForGlobal('AllKeys', timeout);


          And the nice thing about writing it into a helper function like that is that it can be easily used for any global ('_G'-namespace) variable that might take a while to be ready for use. This is the helper function:


          --Wait for a global variable that may not have been initialized yet. If it is nil, wait.
          -- Times out after 'timeout' seconds (or 30s, if only one argument is passed)
          -- Returns the number of seconds which were waited (for debug/monitoring purposes) or false
          -- if the timeout was reached and the variable name still didn't exist.
          function LUTILS.waitForGlobal(globalName, timeout)
              local sleepTimer = 0;
              local LrTasks = import 'LrTasks';
              local timeout = (timeout ~= nil) and timeout or 30;
              while (_G[globalName] == nil) and (sleepTimer < timeout) do
                  sleepTimer = sleepTimer + 1;
              return _G[globalName] ~= nil and sleepTimer or false