Auto Stack JPG's in Front of Raw

[impactsystemsDT]

Hi,

I shoot everything JPG plus Raw.  I have a number of reasons why, but would rather not make this post any longer..... I would like to be able to view everything in my library with all JPG's stacked in front of the RAW with the same file name.  I have been able to Auto-Stack by time (setting time to Zero Seconds) does this.  The Problem is that the NEF's appear first in the stack.  I want the JPG's first, always.  I can figure out any way to do this automatically.  It looks like I would have to manually change which file is first.  I shoot thousands and thousands of files, I need to automate this.  How can I do this?

Secondarily, does anyone know how I could apply a star rating to just the jpg and have it also apply to the raw file?

Thanks

[impactsystemsDT]

Well, I've just found another flaw in this method of autostacking....If I'm shooting sports at 10 fps, it will stack 8-14 shots in one stack.  I'm assuming this is because the lowest level of deliniation is 1 second and it reads 3093 and 3094 as being Zero Seconds apart because they are less than one second apart.  Is there any possible way to correct this and have it recognized hundreds or tenths of a second?

I still can't believe that after all this time there is not a feature that let's you Autostack by file name.  Everything with the same file name (regardless of extension) can be stacked together....seems so simple.

[john beardsworth]

Lightroom's stacking is really designed so the shot that you're going to use is on top of the stack and the others are hidden. This suits a studio workflow where you might shoot a number of alternative frames and only the best of the burst will be used.

As a result of this intention, you will experience varying degrees of dissatisfaction if you use stacking for other workflows such as the one you describe or for grouping HDR/panorama frames. So as you've found, auto stacking isn't controllable by file type, and there's no way to enter metadata and have it automatically cascade down the stack. You're not missing anything - you've hit a brick wall.

John

[richardplondon]

If one wants to sometimes see both the Raw and the JPG, and other times just the JPG, or just the Raw - view filters can do this more easily, on the fly. These can be applied indiscriminately to a folder-based view, or a collection-based view, or a smart collection - unlike stacking. Once a filter preset is named (such as, "JPG-only"), this name is immediately accessible in the Grid, or in the filmstrip, across all LR modules.

Then stacking can be used more for classification and selection purposes - in other words, for grouping and prime-candidacy according to picture subject / content only - which is an interactive matter, of necessity, IMO.

The challenge is, how to achieve that rapidly and robustly with minimal effort.

I would agree that auto-stacking of bursts probably needs a "maximum interval between exposures" rule, rather than a "taken during a given second" rule - though I don't recall using burst mode myself, this century (grin). I'd want to go the other way, and extend that auto-stacking interval to 60 seconds.

A "prefer JPG" or "prefer Raw" option for the auto-stacking feature, sounds to me like a very good idea though. The suggestion has already been made I believe, of a "show JPG" / "show Raw" / "show both" option, for the situation where Raw+JPG pairings are not otherwise set to separate on import.

[cmatts@cmatts.com]

richardplondon wrote:

A "prefer JPG" or "prefer Raw" option for the auto-stacking feature, sounds to me like a very good idea though. The suggestion has already been made I believe, of a "show JPG" / "show Raw" / "show both" option, for the situation where Raw+JPG pairings are not otherwise set to separate on import.

yes! as mentioned by @DaveDuprePhoto Apeture does this exact thing. It treats the raw+jpg as the same file and then you just choose which is the master globally and individually.  I've grown very dependent on it. It making this switch to LR hard.

My immediate response to filtering to just see the jpg file: it is not a good solve. I might as well go back to iPhoto where i had the raw and jpg next to each other.

[impactsystemsDT]

Thanks for the posts.  It seems to be confirming that my preferred workflow is not supported by lightroom.

One thought?  Is there any solution involving conversion to DNG?  It appears that I can store a complete Raw file in the dng AND the embeded JPG preview.  This would work if the actual JPG is the same Large Fine (highest quality) jpg that I am getting when I shoot Raw + JPG.  Is it?  It appears I can embed a "Large" JPG but I'm not sure about the compression settings.  Is it the highest quality and identical to the JPG's I'm getting when I shoot Raw + JPG?

If so, then the last question would be how to actually manipulate these files inside of lightroom.  I suppose I can mess around a bit.  Perhaps I'll compare the file size to determine if the file is the same quality and then see if I can separate if needed in my workflow.

Thoughts?

[john beardsworth]

What's the underlying purpose of the JPEGs? Is there a real need?

[impactsystemsDT]

Thanks for the question.  Here is an example of why

Yesterday - Easter Egg Hunt at my House.  I shot about 300 pictures.  Raw plus Jpg.  I want to go through them and I'll choose about 25-35 shots to document the event and perhaps make a photo book that I can keep for us and send to a few friends.  I want the JPG's for this purpose.  My JPG's generally look quite good.  I do pretty good job with my in camera capture and settings.  So, I want to be able to either spend zero time on these 25-35 shots or perhaps just 30 seconds to 1 minute on some or all to crop, vignette or some very quick adjustments, make my book and I'm done.  My wife is happy, my friends are happy and I have somehting to remeber the event.

Then I will then have 2-5 really nice shots that I'll want to really work on.  I might spend 10-20 minutes on each of these.  I will generally process the raw file for the purpose.  Occassionally the JPG's are just fine, so I'll just process those a bit further.  I'll end of with a 2-5 "Hero" shots of the event.  I might use these to add to the photo book or to print or add to my portfolio.  Often I might just tag them to work tomorrow or next week or even next month when I have more time.

So I want to go select my 25-35 shots from the 300 with a simple pick off the JPG's, then the one's I really want the very best quality and versatility I.d like to be able to click on the stack (JPG in Front, Raw behind total of just 2 shots) and work the Raw file (the Negative). I don't ever have time to process more than a handful of Raw shots from any one shoot and frankly 95% of the time the JPG's look as good as the Raw files even after a fair amount of processing, so I really only need to process the Raw file on certain types of files.  Having said that I am a very competent at processing and use combination of lightroom, photoshop, NIK, OnOne and Topaz.  I just don.t need to do post capture and pre-out sharpening, cloning, healing, content aware fill, gradient filters, advanced color processing etc... on 98% of my files.  The JPG's I create are good enough most all the time.

It seems that many people would want to work this way.  I don't to rate all of the JPG's and the Raw's.  I just want the Raw's (negatives) behind the JPG's so I can dive into them if I need to for the 5% or so of the files that I really want to spend time on.  That doesn't mean I won't use the the other 95% (well more like 10%), it just meanst that the jpg's are just fine to email to friends or make photo books.  Not every file that comes out of Lightroom is supposed to be the cover of Sports Illustrated.  Raw and the JPG that I captured are the same file and it should be treated as such.  Importing as the same file doesn't give you any access to the JPG so I don't see the purpose.

[cmatts@cmatts.com]

impactsystemsDT wrote:

Thanks for the question.  Here is an example of why

Yesterday - Easter Egg Hunt at my House.  I shot about 300 pictures.  Raw plus Jpg.  I want to go through them and I'll choose about 25-35 shots to document the event and perhaps make a photo book that I can keep for us and send to a few friends.  I want the JPG's for this purpose.  My JPG's generally look quite good.  I do pretty good job with my in camera capture and settings.  So, I want to be able to either spend zero time on these 25-35 shots or perhaps just 30 seconds to 1 minute on some or all to crop, vignette or some very quick adjustments, make my book and I'm done.  My wife is happy, my friends are happy and I have somehting to remeber the event.

Then I will then have 2-5 really nice shots that I'll want to really work on.  I might spend 10-20 minutes on each of these.  I will generally process the raw file for the purpose.  Occassionally the JPG's are just fine, so I'll just process those a bit further.  I'll end of with a 2-5 "Hero" shots of the event.  I might use these to add to the photo book or to print or add to my portfolio.  Often I might just tag them to work tomorrow or next week or even next month when I have more time.

So I want to go select my 25-35 shots from the 300 with a simple pick off the JPG's, then the one's I really want the very best quality and versatility I.d like to be able to click on the stack (JPG in Front, Raw behind total of just 2 shots) and work the Raw file (the Negative). I don't ever have time to process more than a handful of Raw shots from any one shoot and frankly 95% of the time the JPG's look as good as the Raw files even after a fair amount of processing, so I really only need to process the Raw file on certain types of files.  Having said that I am a very competent at processing and use combination of lightroom, photoshop, NIK, OnOne and Topaz.  I just don.t need to do post capture and pre-out sharpening, cloning, healing, content aware fill, gradient filters, advanced color processing etc... on 98% of my files.  The JPG's I create are good enough most all the time.

It seems that many people would want to work this way.  I don't to rate all of the JPG's and the Raw's.  I just want the Raw's (negatives) behind the JPG's so I can dive into them if I need to for the 5% or so of the files that I really want to spend time on.  That doesn't mean I won't use the the other 95% (well more like 10%), it just meanst that the jpg's are just fine to email to friends or make photo books.  Not every file that comes out of Lightroom is supposed to be the cover of Sports Illustrated.  Raw and the JPG that I captured are the same file and it should be treated as such.  Importing as the same file doesn't give you any access to the JPG so I don't see the purpose.

agree with your workflow and raw+jpg should be linked as same file (as in aperture)

[cmatts@cmatts.com]

john beardsworth wrote:

What's the underlying purpose of the JPEGs? Is there a real need?

Like picking a film stock i set my camera to give my images a specific look deepening on what/when I'm shooting. It gives me a faster preview of what i had the intended the image to look like when shooting. I won't often remember what i wanted a shot or series of shots to look like after loading the images. I certainly dont want to have to go and "develop" every image just to make selects, and I rather not have all my images look like the default LR raw decoder. Trashing the look you can set in the camera really negates a lot of the creative power newer digital cameras have.

I like the option to just use the jogs the camera makes or use it as a visual ref to process/develop the raw images after i make my selects.

[areohbee]

Here are a couple plugins you may find helpful:

http://www.robcole.com/Rob/ProductsAndServices/StackerLrPlugin

http://www.robcole.com/Rob/ProductsAndServices/RawPlusJpegLrPlugin

Rob

[impactsystemsDT]

Wow Rob! - At first glance these plug-in's look like precisely what I need!  So excited.  I'll report back once I've had a chance to use.

Thanks!

[DaveDuprePhoto]

I have a similar request but for a different reason. I'm about to move around 15,000 images over to LR4 from Aperture. Many of the earlier images are scanned from film JPG masters with minor edits. Starting once I got a camera that did RAW, I have all RAW files. I know I can't move the adjustments over, but what I would like to do is have the file pairs be such that the edited JPG is what is shown, but I have the option to go back to the original RAW or JPG if I want to start over with edits. Normally, I wouldn't do that, but I do occassionaly go back and redo a photo if I get a new tool, or I learn a new technique that could improve it.

Aperture provides a way to swap which file is the master (use JPG as master or use RAW as master). Does LR4 have something similar?

Since some of my files are JPG master + JPG edits, am I limited to the stack approach?

Is there a better way?

Note, I really would rather not keep everything in Aperture around because I'm guessing that will end up causing me headaches if I change things somewhere.

[Michael_Gbg_SE]

I had exactly the same problem, I wanted to stack the raw+jpg files together with the jpg on top.

It is quite surprising that a thing like this can be a problem in a program like Lightroom. My first thought was to select all the jpg files and simply put them on top.

That is how I would expect it to work, but obviously it doesn't.

However, I think I may have stumbled over a solution by accident. Well it is more of a workaround rather than a solution.

I did the following:

I had all my jpg files in a folder and all my raw files (cr2 in my case) in a subfolder.

I used the option to copy the folder into my Lightroom library folder and then "added" the photos, (but I suppose it would be the same with a copy or move operation).

First I imorted/added all the jpg files into Lightroom. Then I manually moved the raw files to the same folder as the jpg files and made another import/add of the raw files.

After that I just stack according to capture time and the jpg file comes first, i.e. on top.

I suppose that when you import the raw+jpg at the same time, the raw file is imported first and therefore goes on top.

I haven't tested this thoroughly, but it seems to do the trick.

[photomonk_]

It works. Slightly different, but same idea.

Having already all RAW+jpeg imported, I just select RAW by filter (metadata->filetype->RAW), than removed them from catalog (not delete, just remove!) and than synchronised that folder.

After that stacking work as I want - raw+jpeg, jpeg on top.

[photomonk_]

Now I'm looking for way to set an attribute (for example reject or pick flags) on both RAW+jpeg when working with collapsed stacks...

[areohbee]

No way to do it in Lightroom without expanding stacks (and selecting both), except via plugin.

[photomonk_]

Rob, can you name a plugin that actually does this (at least flags)? Plugins that are listed above look like they can not. First is M$ only

[areohbee]

Hmm - Indeed: RawPlusJpeg plugin won't transfer metadata down through collapsed stack, yet - bummer: on my to-fix list..

In the mean time, you can use:

* RelativeAntics (free by me) - I just checked the latest version - it transfers (e.g. pick flag) even if stack collapsed.

(note: you must edit advanced settings (preset manager section of plugin manager - via drop-down menu) and change:

--_inclMeta["pickStatus"] = true

to

_inclMeta["pickStatus"] = true

(i.e. remove the '--' at the front, thus "uncommenting" the line) to enable pick-flags to be transferred - sorry about the hassle - let me know outside the forum if problems doing that. The good news: you can see exactly what will / will not be transferred, and you can customize the others in like fashion (include or exclude).

* SearchReplaceTransfer (by John Beardsworth - not free).

[photomonk_]

got it. I'll give it a try this evening (14:37 here ). thanks!

[areohbee]

Where are you? Iraq, Iran, Kuwait?

(I computed UTC + 03:00 Hours)

[photomonk_]

Russia

Plugin requests catalog update... I'm kinda afraid. Is it safe? (Mac OS here 10.1.1, LR 5.6)

[areohbee]

Yes. plugin uses a little bit of custom metadata to support operation, but it won't effect anything else.

Out of curiosity, which metadata items are you wanting to send down in the stack(s), and which photos in the stack are to receive the metadata?

For example, are virtual copies viable recipients? how about videos.. And are there any metadata items you would NOT want to be "sent down" etc.

The reason I ask, is I have another script (not a plugin) you may prefer (it's not released, yet):

It looks a little daunting because there is a big square of lua code staring at you, but in a way it's simpler than Relative Antics (because it only has one purpose), and I can help customize (create a "favorite" which does what you want).

PS - Daylight savings there?

[photomonk_]

My initial situation is just like in post #6. Shooting a lot, give out all "good" with LR template watermarking (often), retouching few "best" shots (slightly LR, mostly Photoshop). Recently switched from Nikon D800 to Fujifilm X-Pro1 and jpegs are just good enough that I can give them away "AS IS", thats where I got interested in stacking again. Incamera jpegs from Nikon always needed tweaks in skintones (my solution - custom built camera calibration profiles with X-Rite Colour Checker Passport + tweaking them in DNG profile editor).

Attributes that I see useful for me to be send down in stack right now:

(note: talking about stacking just RAW+jpeg with jpeg on top here, not series of RAWs)

- flags

- ratings

- exposure (maybe all attributes from Basic section, but usually they are OK from camera)

- crop & straighten

Working with wedding photos looks like completely different scenario, where set of attributes is much wider (Basic, Tone curve, HSL/Colour/B&W/Split toning/Effects). No RAW+jpeg stacks, but RAW only, as I see from now (not sure here since started working with Fuji) . Honestly, thought I'm constantly looking for more efficient workflow, right now I'm OK with just syncing settings between photos (wedding case), cause I need to see all photos to be modified (habit?). Need to give it a try.

See no use of stacking video right now, making few of them - just for fusion / slideshow. Sync is enough for today.

I do have IT background, code chunks do not frighten me Yet December leaves much less time for tests and changes - lots of work...

Daylights savings  - yes, and they are a mess here ))) Some egghead made such a decision this year, that google calendar went crazy, cause there were no appropriate timezone settings at moment is was announced ))) Syncing phone, laptop and online calendars... smiling right now, but at that moment it wasn't that funny.

[areohbee]

photomonk_ wrote:

- flags

- ratings

- exposure (maybe all attributes from Basic section, but usually they are OK from camera)

- crop & straighten

Well, you've got 3 types of "metadata" there, each with it's own set of "challenges".

Actually, flags and ratings are a piece of cake, because the plugin SDK has direct support.

Exposure is a little trickier, but certainly doable (there is "experimental" support in SDK - but care must be taken..).

Crop & straighten (and orientation) are especially tricky: there is no support for them in the SDK, which means one has to do things like:

* save xmp

* manipulate xmp, e.g. via exiftool

* re-read xmp.

Lr5 has undocumented methods for saving/reading xmp. They're "usable" (in Windows anyway) but there may be more caveats than I know, e.g. on Mac (and I know of a few). In the past I have saved metadata programmatically using osascript on Mac, to simulate keystrokes, but the newer Macs are getting more restrictive and so it often doesn't work on Mac, and there is unfortunately no keyboard accelerator for reading metadata on Mac, which means that technique only works in Windows (via AutoHotkey instead of osascript).

Anyway, both RelativeAntics and RawPlusJpeg have been programmed to do the xmp thing, however RawPlusJpeg won't do it (yet) if recipient photo is a stack underling. Note: jb's SearchReplaceTransfer does not support crop/straighten nor orientation at all.

Anyway, I dunno whether I want to add xmp manipulation to the "Send Metadata Down In Stack" script or not - I may just re-work RawPlusJpeg - we'll see. If you do opt for RawPlusJpeg over Relative Antics (or the aforementioned "script"), you'll need to get used to frequently expanding and re-collapsing stacks (until it is enhanced..).

I wish Adobe would just add the support we need to do stuff like this, but their attention is mostly elsewhere these days.

Better also if they would go ahead and tackle raw+jpeg workflow themselves so no plugin needed..

Oh well, I'd probably understand better if I was ga-ga over iPhones and Lr Mobile..

PS - Sorry for so much plugin-specific info: if you want to talk more about how to optimize workflow using my plugins, you can contact me offline - thanks.

Cheers,

Rob