Skip navigation
davidpope007
Currently Being Moderated

"Date Modified" for all files being changed if "Automatically write to XMP" is on

Aug 9, 2011 9:21 PM

Recently upgraded from LR3 (v3.4.1) from LR2 on OSX 10.6.8 and have always had Catalog Settings > Automatically write changes into XMP turned on.

 

When browsing existing JPG files in my Library (no Develop changes, no keywording, no Presets, no Import), LR3 is writing to disk — i.e., when I look at files in Finder, almost every viewed file’s “Date Modified” is being set to today’s date and time. (It actually creates a .swp file, then changes it's name back to .jpg)

 

This is really bad, as it's making it impossible for me to use Finder to figure out when I last worked with a file, it is triggering needless Time Machine and Backblaze backups, and unnecessarily churning my disk.

 

If I turn off "Automatically write..." this behavior stops. Per David Marx at thelightroomlab.com, I tried turning off this preference, manually doing a "Save Metadata to File" for all files, letting that complete, then turning the preference back on. This does not solve the problem.

 

Per a suggestion at photoshop.com, I used ExifTool to see what changes LR was writing to a sample file; from the diff below, you can see that LR is adding a bunch of new fields as well as moving other fields around. But my point is that LR3 should never overwrite a file on disk if all I am doing is browsing thru them.

 

Is anyone else seeing this? Any ideas would be greatly appreciated!


-- David

diff Exif5609_original Exif5609_update
2c2
< FileName: DSC_5609_original.JPG
---
> FileName: DSC_5609_update.JPG
5,6c5,6
< FileModifyDate: 2009:11:27 21:32:54-08:00
< FilePermissions: rwxr-xr-x
---
> FileModifyDate: 2011:08:07 22:06:47-07:00
> FilePermissions: rw-r--r--
27a28,29
> ShutterSpeedValue: 1/200
> ApertureValue: 7.1
55d56
< SerialNumber: 3209521
75d75
< Lens: 18-200mm f/3.5-5.6
185,188d184
< UserComment:
< SubSecTime: 00
< SubSecTimeOriginal: 00
< SubSecTimeDigitized: 00
211a208,299
> XMPToolkit: Adobe XMP Core 5.2-c004 1.136881, 2010/06/10-18:11:35
> CreatorTool: Ver.1.00
> MetadataDate: 2011:08:07 22:06:47-07:00
> SerialNumber: 3209521
> LensInfo: 18-200mm f/3.5-5.6
> Lens: 18.0-200.0 mm f/3.5-5.6
> ImageNumber: 26634
> RawFileName: DSC_5609.JPG
> SavedSettingsName: Import
> SavedSettingsType: Snapshot
> SavedSettingsParametersVersion: 6.4.1
> SavedSettingsParametersProcessVersion: 5.0
> SavedSettingsParametersWhiteBalance: As Shot
> SavedSettingsParametersIncrementalTemperature: 0
> SavedSettingsParametersIncrementalTint: 0
> SavedSettingsParametersExposure: 0.00
> SavedSettingsParametersShadows: 0
> SavedSettingsParametersBrightness: 0
> SavedSettingsParametersContrast: 0
> SavedSettingsParametersSaturation: 0
> SavedSettingsParametersSharpness: 0
> SavedSettingsParametersLuminanceSmoothing: 0
> SavedSettingsParametersColorNoiseReduction: 0
> SavedSettingsParametersChromaticAberrationR: 0
> SavedSettingsParametersChromaticAberrationB: 0
> SavedSettingsParametersVignetteAmount: 0
> SavedSettingsParametersShadowTint: 0
> SavedSettingsParametersRedHue: 0
> SavedSettingsParametersRedSaturation: 0
> SavedSettingsParametersGreenHue: 0
> SavedSettingsParametersGreenSaturation: 0
> SavedSettingsParametersBlueHue: 0
> SavedSettingsParametersBlueSaturation: 0
> SavedSettingsParametersFillLight: 0
> SavedSettingsParametersVibrance: 0
> SavedSettingsParametersHighlightRecovery: 0
> SavedSettingsParametersClarity: 0
> SavedSettingsParametersDefringe: 0
> SavedSettingsParametersHueAdjustmentRed: 0
> SavedSettingsParametersHueAdjustmentOrange: 0
> SavedSettingsParametersHueAdjustmentYellow: 0
> SavedSettingsParametersHueAdjustmentGreen: 0
> SavedSettingsParametersHueAdjustmentAqua: 0
> SavedSettingsParametersHueAdjustmentBlue: 0
> SavedSettingsParametersHueAdjustmentPurple: 0
> SavedSettingsParametersHueAdjustmentMagenta: 0
> SavedSettingsParametersSaturationAdjustmentRed: 0
> SavedSettingsParametersSaturationAdjustmentOrange: 0
> SavedSettingsParametersSaturationAdjustmentYellow: 0
> SavedSettingsParametersSaturationAdjustmentGreen: 0
> SavedSettingsParametersSaturationAdjustmentAqua: 0
> SavedSettingsParametersSaturationAdjustmentBlue: 0
> SavedSettingsParametersSaturationAdjustmentPurple: 0
> SavedSettingsParametersSaturationAdjustmentMagenta: 0
> SavedSettingsParametersLuminanceAdjustmentRed: 0
> SavedSettingsParametersLuminanceAdjustmentOrange: 0
> SavedSettingsParametersLuminanceAdjustmentYellow: 0
> SavedSettingsParametersLuminanceAdjustmentGreen: 0
> SavedSettingsParametersLuminanceAdjustmentAqua: 0
> SavedSettingsParametersLuminanceAdjustmentBlue: 0
> SavedSettingsParametersLuminanceAdjustmentPurple: 0
> SavedSettingsParametersLuminanceAdjustmentMagenta: 0
> SavedSettingsParametersSplitToningShadowHue: 0
> SavedSettingsParametersSplitToningShadowSaturation: 0
> SavedSettingsParametersSplitToningHighlightHue: 0
> SavedSettingsParametersSplitToningHighlightSaturation: 0
> SavedSettingsParametersSplitToningBalance: 0
> SavedSettingsParametersParametricShadows: 0
> SavedSettingsParametersParametricDarks: 0
> SavedSettingsParametersParametricLights: 0
> SavedSettingsParametersParametricHighlights: 0
> SavedSettingsParametersParametricShadowSplit: 25
> SavedSettingsParametersParametricMidtoneSplit: 50
> SavedSettingsParametersParametricHighlightSplit: 75
> SavedSettingsParametersSharpenRadius: +1.0
> SavedSettingsParametersSharpenDetail: 25
> SavedSettingsParametersSharpenEdgeMasking: 0
> SavedSettingsParametersPostCropVignetteAmount: 0
> SavedSettingsParametersGrainAmount: 0
> SavedSettingsParametersLensProfileEnable: 0
> SavedSettingsParametersLensManualDistortionAmount: 0
> SavedSettingsParametersPerspectiveVertical: 0
> SavedSettingsParametersPerspectiveHorizontal: 0
> SavedSettingsParametersPerspectiveRotate: 0.0
> SavedSettingsParametersPerspectiveScale: 100
> SavedSettingsParametersConvertToGrayscale: False
> SavedSettingsParametersToneCurveName: Linear
> SavedSettingsParametersCameraProfile: Embedded
> SavedSettingsParametersCameraProfileDigest: D6AF5AEA62557FCE88BC099788BBD3CC
> SavedSettingsParametersLensProfileSetup: LensDefaults
> SavedSettingsParametersToneCurve: 0, 0, 255, 255
> IPTCDigest: d41d8cd98f00b204e9800998ecf8427e
228,230d315
< SubSecCreateDate: 2009:11:27 21:32:54.00
< SubSecDateTimeOriginal: 2009:11:27 21:32:54.00
< SubSecModifyDate: 2009:11:27 21:32:54.00

 
Replies
  • Currently Being Moderated
    Aug 9, 2011 9:40 PM   in reply to davidpope007

    Much discussed in other threads. If the XMP being written to is /in/ the file (as it is for non-proprietary files) as opposed to a sidecar file, then the file IO being done is going to update the modified timestamp.

     

    Lr by and large does not write out anything to the referenced files unless you make a gesture that indicates you want this to happen, the great exception being what happens when this (by default, disabled) setting is enabled. In that case, every metadata change is an implicit gesture to write changes to the XMP block.

     

    The follow-on issues you describe are some of the side-effects of doing this.  The other alternative is for Lr to touch the file back to the original timestamp, which has its own problems.

     

    Consider, first, whether you need this setting enabled. Do you need to transmit the state of the image to other XMP aware apps regularly?  If so, then you have to balance this with other considerations.

     

    But, like I said, this is much discussed here. Perhaps someone else has other workarounds.

     

    But, what you are seeing is expected, as far as I know.

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 9, 2011 11:06 PM   in reply to davidpope007

    You want to rely on the system-level file modification date as part of your workflow and that won't work because LR also makes changes to the contents of the file unrelated in time to your changes.  It's not a bug or a horrible design decision, just a difference of opinion about how things should work.  I'm sure Adobe had a good reason to do things this way, not just the mess people up. 

     

    It would make sense that LR would want to update the fields in the file, itself, for performance reasons or for code simplicity, so it's easier to synchronize the folder or otherwise check if there have been any outside changes by another program and not have to do the metadata version upgrade in memory every time the file is looked at. 

     

    I'm surprised that LR doesn't update the metadata in every file in your database, slowly, in the background, not just files you click on, but maybe it was a performance hit every time people upgraded their LR version so Adobe chose to do it different and avoid the bad press about performance being bad in the new version.

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 10, 2011 8:03 AM   in reply to davidpope007

    davidpope007 wrote:

     

    Then when LR3 loaded my old LR2 images into memory, it "dirtied" the in-memory copy of the file by adding in these new LR3 XMP fields. Then, because I had "Automatically write XMP" on, it said "I better write these changes to disk".

     

    Yuck. As a former software engineer, this is very bad software engineering.

     

    It should wait until the user dirties the file (via Develop, keywords, etc.) before presuming to add a bunch of metadata fields that are unique to the new version of LR3.

     

     

    Well, I'm a current software developer, and this is, really, a perfectly reasonable thing to do. It is a reasonable trade-off for a convenient feature required by a small subset of users.

     

    Yes, in most cases the in-memory copy should "never" be dirtied unless the user makes a gesture of some sort, but like I said earlier, this option (once set by the user) sets up the situation where this gesture becomes implicit. This is a clear trade-off for the sake of convenience. And if the XMP is out of date and needs to be updated en masse, so be it.

     

    The fact is, there is no easy way around this. Do we save up /every/ dirty buffer somehow until you make a gesture that /might/ require the XMP to be up-to-date before acting on that gesture? Now we have to worry about unflushed buffers if something goes wrong and the app exits. Do we save the buffers to the DB? Now we have to block some calls to make another blocking call to flush some or all of those to DB, and then write some or all of it out to one or more files. In what order? What if there is a gesture to have X files with up-to-date XMP and some or all of those are in unflushed buffers, unflushed DB writes or we have to wait for the DB.

     

    As you can see, this is a transactionality nightmare, and the easiest and safest thing to get what the user wants (i.e., up-to-date XMP for the purpose of talking to a third-party XMP aware app) is to simply update the sidecar or XMP block in an atomic manner using the correct file IO. The file will have to change at some point, so it may as well be now.

     

    In fact, the only solution that isn't a tranactioanlity nightmare, and one that is a feature request already, is to force Lr to always use sidecar files, even for filetypes with XMP blocks. I wouldn't hold my breath for this one, though.

     

    Regardless, the first thing we should do is evaluate if we even need this option enabled. If so, then we need to evaluate why we are relying on the file modified timestamp, because this is usually a good sign that something in the workflow is not ideal. I would use this opportunity to inquire into what, exactly, you need from the app, and why the current behaviour is causing you a problem. Because if the answer is primarily that you don't like the inelegance of the solution, then I'm afraid you aren't going to get much sympathy. If you need the functionality this option gives you, then there are side-effects that you will have to accept for this convenience.

     

    Anyway, I'm not expecting to convince you otherwise, since you seem to have made your mind up. I will finish by saying that decades of experience has taught me that absolute rules are rarely absolute, and the entire software industry is a trade-off because users require functionality that can run contrary to received wisdom. The type and extent of those compromises only change with the target audience for the app.

     
    |
    Mark as:

More Like This

  • Retrieving data ...

Bookmarked By (0)

Answers + Points = Status

  • 10 points awarded for Correct Answers
  • 5 points awarded for Helpful Answers
  • 10,000+ points
  • 1,001-10,000 points
  • 501-1,000 points
  • 5-500 points