[export plugin] Bug in LrHttp.PostMultipart and LrHttp.Post

Interesting.  Can you post an example of your export code?  There are plenty of of export plug-ins that post images to websites and work.  I have tested my latest export plug-in with files up to 80Mb so I wonder if it is something specific about your code or the file you are testing with.

I know there are a few LR to Gallery plugins out there that do work. Mine are only going up to 5MB as that's the server limit for those services.. no issues though.

Hi,

I am the developper of the plugin, so I guess I can bring a little more information here. As far as I know, most users don't have any problem to upload large pictures, but I know some can not, for reasons I can not explain. I don't have the capacity nor time to test all configurations of my users, however, I try to stay as OS agnostic as I can, and for the moment, there is nothing I know of that is OS specific in my code.

Here is some code to explain how it all works :

The loop to process generated pictures :

    for i, rendition in exportContext:renditions{ stopIfCanceled = true } do
    
        -- Get next photo.
        local photo = rendition.photo
        local success, pathOrMessage = rendition:waitForRender()
        
        -- Check for cancellation again after photo has been rendered.
        if progressScope:isCanceled() then
            break
         end
        
        if success then

            local filename = LrPathUtils.leafName( pathOrMessage )

            local caption = ''
    
            -- Set the caption
            if captionSetting == 'none' then
                caption = ''
            elseif captionSetting == 'filename' then
                caption = filename
            elseif captionSetting == 'title' then
                photo.catalog:withCatalogDo( function() 
                    caption = photo:getFormattedMetadata ( 'title' )
                end ) 
            elseif captionSetting == 'caption' then
                photo.catalog:withCatalogDo( function() 
                    caption = photo:getFormattedMetadata ( 'caption' )
                end ) 
            end    
            
            -- upload file to gallery
            success = GalleryRemoteProtocol.uploadImage( serverId, albumName, pathOrMessage, caption )
            
            if success ~= '0' then
                -- if we can't upload that file, log it.  For example, maybe user has exceeded disk
                -- quota, or the file already exists and we don't have permission to overwrite, or
                -- we don't have permission to write to that directory, etc....
                table.insert( failures, filename )
            end
                    
            -- When done with photo, delete temp file. There is a cleanup step that happens later,
            -- but this will help manage space in the event of a large upload.
            LrFileUtils.delete( pathOrMessage )
        end

And here is the function to upload the file :

function GalleryRemoteProtocol.uploadImage(serverId, album, imagePath, caption)
    log:trace("Calling GalleryRemoteProtocol.uploadImage( "..prefs.serverTable[serverId].label..", "..album..", "..imagePath..", "..caption.." )")
    
    -- get server
    local server = prefs.serverTable[serverId].server
    
    local filename = LrPathUtils.leafName( imagePath )
    
    local response, debug, content
    if GalleryRemoteProtocol.galleryVersion == '1' then
        -- construct operation
        content =
            { { name='cmd', value='add-item' },
              { name='protocol_version', value='2.0' },
              { name='set_albumName', value=string.encodeEntities( album ) },
              { name='caption', value=string.encodeEntities( caption ) },
              { name='userfile_name', value=filename },
              { name = 'userfile',
                fileName = filename,
                filePath = imagePath,
                contentType = 'multipart/form-data'
              }
            }
    else
        -- construct operation
        content =
            { { name='g2_form[cmd]', value='add-item' },
              { name='g2_form[protocol_version]', value='2.0' },
              { name='g2_form[set_albumName]', value=album },
              { name='g2_form[caption]', value=caption },
              { name='g2_form[force_filename]', value=filename },
              { name='g2_authToken', value=GalleryRemoteProtocol.authToken },
              { name = 'g2_userfile',
                fileName = filename,
                filePath = imagePath,
                contentType = 'multipart/form-data'
              }
            }
    end
    -- send POST request to the gallery server
    response, debug = LrHttp.postMultipart( GalleryRemoteProtocol.getGalleryRemoteURL(server), content )
    
    -- parse reponse into easy to digest table format
    responseTable = parseGalleryResponse( response )
        
    -- get server response status
    local serverStatus = getResponse( responseTable, 'status' )
    
    return serverStatus
end

As you can see, this is pretty dumb code.

By the way, I had the chance to have Eric Scouten review my code. I got valuable information and one fix. However, he did not mention any potential problem in the upload section.

If this can help...

I did forward your message to one of our testers. He took the time to set up a Gallery server internally and create a reproducible test case that demonstrates a failure to upload large files with this plug-in. I haven't had the time to investigate this bug yet, but it is on my plate for LR3. (Sorry, can't provide info on when that will be available.)

A possible cause of this issue is the wrong content type in your MIME chunk for the image file:

{ name = 'g2_userfile',
  fileName = filename,
  filePath = imagePath,
  contentType = 'multipart/form-data'
}

Try changing it as follows: contentType='image/jpeg'

Vladimir Vinogradsky wrote:

 

A possible cause of this issue is the wrong content type in your MIME chunk for the image file:

{ name = 'g2_userfile',
  fileName = filename,
  filePath = imagePath,
  contentType = 'multipart/form-data'
}

 

Try changing it as follows: contentType='image/jpeg'

Right !

I changed this and made some tests. It seams OK.

However, I don't think this is the reason of the initial problem

Eric, thanks for the information!

I think, this is going to be very tricky to nail down and maybe to reproduce. The described problem seams to be at the TCP level, so the Gallery server is important there.

Bpub, can you tell :

• Which OS/machine is used to run Gallery
• Which webserver is used (and version)
• Which PHP engine is used (and version)

Maybe we won't be able to reproduce the same conditions. I have a similar case with someone else running Gallery 1, and can not upload large pictures, but I can upload the same pictures on his Gallery installation from my Windows XP laptop. Weird...

Eric, if you need a test account on a Gallery 2, I can provide you with one. Maybe Bpub could do too, it might help.

Arnaud, I'll keep this in mind, but at this point, our tester has indicated that he was able to create a pretty solid reproducible case in-house. Hopefully that will be sufficient to figure out what's going wrong.

Message was edited by: Eric Scouten (correcting a typo)

Additionnal information : I reinstalled Windows last week-end on my machine. This time I decided to give Windows 7 a try (after XP).

I tested the plugin and LR on Windows 7 and almost everything seems to be fine, except I cannot create any album. the Plugin seams to stop in LrHttp.PostMultipart (the last try I have is from the GetGalleryRemoteUrl within the parameters. So I think, something gets stuck in PostMultipart. However, I managed to upload very large pictures to the same Gallery using also PostMultipart.

By the way here is the code of the function to create an album. As you can see, this is not rocket science

-- Add a new album to the server 
function GalleryRemoteProtocol.addAlbum(serverId, parentAlbum, albumName, albumTitle, albumDescription)
    log:trace("Calling GalleryRemoteProtocol.addAlbum( "..prefs.serverTable[serverId].label..", "..parentAlbum..", "..albumName..", "..albumTitle..", "..albumDescription.." )")
    
    -- get server
    local server = prefs.serverTable[serverId].server
    
    local response, debug
    if GalleryRemoteProtocol.galleryVersion == '1' then
        response, debug = LrHttp.get( GalleryRemoteProtocol.getGalleryRemoteURL(server) ..
            "?cmd=new-album&protocol_version=2.1&set_albumName=" .. parentAlbum ..
            "&newAlbumName=" .. albumName ..
            "&newAlbumTitle=" .. albumTitle ..
            "&newAlbumDesc=" .. albumDescription )
    else
        -- construct operation
        local operation =
            { { name='g2_form[cmd]', value='new-album' },
              { name='g2_form[protocol_version]', value='2.1' },
              { name='g2_form[set_albumName]', value=parentAlbum },
              { name='g2_form[newAlbumName]', value=albumName },
              { name='g2_form[newAlbumTitle]', value=albumTitle },
              { name='g2_form[newAlbumDesc]', value=albumDescription },
              { name='g2_authToken', value=GalleryRemoteProtocol.authToken },
            }    
        
        -- define headers
        local headers =
            { { name='Content-Encoding', value='utf-8'},
            }
        
        -- send POST request to the gallery server
        response, debug = LrHttp.postMultipart( GalleryRemoteProtocol.getGalleryRemoteURL(server), operation, headers )
    end
    -- parse reponse into easy to digest table format
    responseTable = parseGalleryResponse( response )
    
    -- get server response status
    local serverStatus = getResponse( responseTable, 'status' )
    
    return serverStatus, getResponse( responseTable, 'album_name' )
end

My mistake on this one...

In the headers, I specified "name" instead of "field", LR 2.0 to 2.3 seamed OK with it, but LR 2.4 and LR 2.5 could not execute the LrHttp.postMultipart witouth dying with no error message.

This one is corrected and not related to the initial problem.

Is this issue already confirmed or is there a solution/workaround for it.

I just discovered that my plugin has problems with 100+ MB photos.

One user gets an "not enough memory" error message. While I get (with the same photo) a 'An internal error occured: LuaRunException'

This occurs in the PostMultipart call and after that any PostMultipart will fail unit lightroom is closed (ie force a kill of the lightroom process) and restarted.

Any suggestions?

Paul