Lightroom can really get bogged down when doing large numbers of catalog updates as a single transaction.

I suggest limiting to 1000 photos a whack. I think Lr4 is worse than Lr3 in this regard, or at least it is for me anyway...

Rob

You're welcome ocroquette.

Only way I know is to return from the "with-do" function, and then go back in...

Hint: Use withWriteAccessDo instead of the "prolonged" version.

ocroquette wrote:

 

I was almost done, but now I have to refactor my code to process batch of 1.000 files instead of processing the 30000 files at one, and that's not a trivial task

It's a pisser alright, and although not trivial, it's not too difficult either.

I've been lobbying with Adobe to remedy this. It's one thing for it to take a long time, but a worse problem is that it can over-consume ram and hang the system.

I decided to build the incremental catalog updating into my core methods (not yet released):

http://forums.adobe.com/thread/1009149

Even splitting the update into chunks, Lightroom gets slower and slower as it goes. For me, it usually makes it through if I leave it run long enough, but it's a mystery why it gets sooooooo slow after a time...

Rob

Very possibly.

I've been really stoked with my new multi-phase catalog accessor - strongly recommend writing something like it for yourselves (see source code in other thread, or contact me for fresh stuff).

Here's an example from my most recent plugin:

local s, m = cat:update( 10, "Assure Folder Collection Set", function( context, phase ) -- 10 is seconds to get in.

    -- function can take parameters, but not used here.

    if phase == 1 then

        set = catalog:createCollectionSet( folder:getName(), parent, true )

        return false -- keep going

    elseif phase == 2 then

        leafPhotos = folder:getPhotos(false)

        if #leafPhotos > 0 then

            return false

        else

            return true -- done

        end

    elseif phase == 3 then

        coll = catalog:createCollection( str:fmt( '[^1]', folder:getName() ), set, true )

        app:logVerbose( "Created/assured \"Folder Leaf Photos\" collection ^1 in '^2'", folder:getName(), set:getName() )

        return false

    elseif phase == 4 then

        local collPhotos = coll:getPhotos()

        if #collPhotos > 0 then

            local collPhotoSet = tab:createSet( collPhotos )

            for i, photo in ipairs( leafPhotos ) do

                if not collPhotoSet[photo] then

                    toAdd[#toAdd + 1] = photo

                end

            end

        else

            toAdd = leafPhotos

        end

        if #toAdd > 0 then

            coll:addPhotos( toAdd )

        end

        return true

    end

end )

Note: each phase is a new transaction.

In your case Oliver:

local keyword

local children

local s, m = cat:update( 10, "Keyword business", function( context, phase )

     if phase == 1 then

          keyword = catalog:createKeyword( "Test", {}, true, nil )

          return false -- continue to 2nd phase.

    elseif phase == 2 then

          children = keyword:getChildren()

          -- do something to catalog with children

          return true -- or nil.

     end

end )

This would not make sense if nothing further involved catalog writes (in which case just exiting a simple single-phase with-do method would suffice), but you get the idea, no?

R