Initial sync gets stuck

Report · Nov 18, 2017

Lightroom CC Version 1.0.1 on macOS 10.13.2 Build 17C79a

I have a critical issue with the initial photo sync in Lightroom CC:

I started by migrating my old Lightroom CC Classic catalogue with 16000 images. Because I didn't have enough disk space locally I had to configure LR CC to store the migration files on an external USB drive. Migration itself worked fine and didn't show any errors. All my original files were transferred to my USB drive and the upload of these files started. Because these images take a while to upload I can't sync them in one go.

I now have the problem that Lightroom CC does not continue syncing when I restart Lightroom with a mounted external drive. It is just sitting there causing slight network activity and is using ~150% CPU. This goes on for multiple hours and does not upload any images (remaining image count does not change).

I checked the system logs and when this happens I get a lot of messages from "Adobe Lightroom CC Helper" like:

Task <BC1B4F5E-5BB2-45D3-8D29-72695825B9CC>.<880> finished with error - code: -999

and also

WFOAuth2SessionImp ERROR error on URL "https://storage.adbephotos.com/v2/catalogs/267e24e5f8e54af589a3c3d9c73eb83f/assets/0ac4e086790049408..."
data = while (1) {}
{"code":1005,"description":"Input Validation Error","errors":{"parts":["can't be blank"]},"subtype":"InputValidationError"}
headers = {
  ["Access-Control-Allow-Credentials"] = "true",
  ["Access-Control-Allow-Methods"] = "GET,HEAD,PUT,DELETE,POST,OPTIONS",
  ["Access-Control-Max-Age"] = "60",
  Connection = "keep-alive",
  ["Content-Length"] = "136",
  ["Content-Type"] = "application/json",
  Date = "Sat, 18 Nov 2017 09:42:55 GMT",
  Server = "APIP",
  Status = "400 Bad Request",
  Via = "1.1 ab10c9d2f12520e5d43e0b69bfa09a3c.cloudfront.net (CloudFront)",
  ["X-Account-Id"] = "5fe19fcef66ce5ba010d4ea2fac9322b",
  ["X-Amz-Cf-Id"] = "BBG_Lxq8ZIkn<…>

I can supply you with more logs on demand.

To me it looks like an issue with the helper and it seems as if I have found a way to work around this issue:

My suspicion is that Lightroom Helper is not working properly if it is started with the mounted drive. With the following procedure I get it to continue to sync:

* start Lightroom CC

* unmount external drive

* kill Lightroom Helper

* wait a few seconds for it to restart and the red banner appears in LR

* mount drive again => syncing continues.

Correction: this doesn't help either. Once it was apparently uploading more data but the number of pending photos didn't decrease. Now it's back to its regular behaviour (not doing anything).

Can you please have technicians have a look at this issue? To me this does not look like a very reliable way to store all my images and I'm seriously reconsidering my decision to use Creative Cloud as photo backup! As I said I happily help with logfiles and trying out things - I still have ~13000 pictures to sync.

Message was edited by: Christian Stangier

Report · Nov 20, 2017

After some more investigation I looked through my pending images on the external drive. I found out that about 2/3 of the images there have a size of 0 bytes. So the migration process didn't finish copying the files to the drive. However, when it finished it reported no errors whatsoever.

After replacing all 0 byte files with their originals from my old catalogue the syncing continued.

Until the next problem occured...

I am now at a point where "Adobe Lightroom CC Helper" uses all the available 16GB of RAM and swap space. It doesn't do anything (number of pending images is stuck.. again!) and only consumes all my RAM. CPU utilization of Lightroom CC is constantly at 100%.

When I force kill the helper the same issue repeats. Pending count doesn't decrease. So to me it looks as if it's at some point where it runs into an infinite loop. This even happens when syncing is paused manually or because the external drive is not mounted. So it does not seem to be related to the sync but to the database.

I feel like an alpha tester. This software is clearly not ready for the market!

Additionally I must say I'm quite unhappy with the "support" I am receiving. There is no other means than this forum and apparently noone here feels responsible.

Report · Nov 20, 2017

Ok. I found out that the helper is still doing a bit of uploading until it has used up so much RAM that it's useless.

When the helper process is killed, it is restarted by Lightroom CC. I now created a script that monitors the helper's RAM usage and kills it if it exceeds a certain value (insert facepalm emoji here).

The number of pending photos to sync is decreasing - slowly but surely.

This is my script. I hope this is reducing the pain for other people running into similar issues:

#!/bin/sh
if [ $# -ne 1 ];
then
    echo "Invalid number of arguments. usage: kill-ps-helper.sh (Max used RAM in %). Example: ./kill-ps-helper.sh 10"
    exit 0
fi
while true;
do
    PID=$(pgrep "CC Helper")
    if [ -n "$PID" ]; then
        RAM_USAGE=$(ps -p $PID -o '%mem' | sed 1d)
        ROUNDED_RAM_USAGE=${RAM_USAGE%%.*}
        echo "$(date): RAM Usage: $ROUNDED_RAM_USAGE%"
        if [ $ROUNDED_RAM_USAGE -gt $1 ]; then
            echo "$(date): Killing!"
            kill -9 "$PID"
        fi
    else
        echo "$(date): Helper not running"
    fi
    sleep 5
done

Output is the following:

Mon Nov 20 18:49:10 CET 2017: RAM Usage: 6%

Mon Nov 20 18:49:15 CET 2017: RAM Usage: 7%

Mon Nov 20 18:49:20 CET 2017: RAM Usage: 7%

Mon Nov 20 18:49:25 CET 2017: RAM Usage: 8%

Mon Nov 20 18:49:30 CET 2017: RAM Usage: 8%

Mon Nov 20 18:49:35 CET 2017: RAM Usage: 9%

Mon Nov 20 18:49:40 CET 2017: RAM Usage: 10%

Mon Nov 20 18:49:45 CET 2017: RAM Usage: 10%

Mon Nov 20 18:49:50 CET 2017: RAM Usage: 12%

Mon Nov 20 18:49:50 CET 2017: Killing!

Mon Nov 20 18:49:55 CET 2017: Helper not running

Mon Nov 20 18:50:00 CET 2017: RAM Usage: 0%

Mon Nov 20 18:50:05 CET 2017: RAM Usage: 1%

Mon Nov 20 18:50:10 CET 2017: RAM Usage: 2%

Mon Nov 20 18:50:15 CET 2017: RAM Usage: 3%

Mon Nov 20 18:50:20 CET 2017: RAM Usage: 3%

I'll keep this thread updated with my progress. Kind of my sad Lightroom syncing diary.

Report · Dec 19, 2017

I'm with you. I tried syncing thousands of photos from my laptop while on a trip and constantly had problems. Now, a month and a version update later, I'm trying to sync my desktop catalog with over 12,000 photos in it, and I'm stuck at 10,929 left to sync. No way found for me to get past it.

Adobe Community

Initial sync gets stuck