Auto Mask not working on HDR images

[digitaloxygen]

Issue:

Auto Mask does not work on HDR .dng images. It behaves as if auto make is not turned on however it works as expected when used on one of the original images.

See this video for an example: Dropbox - LR Classic CC Auto Mask HDR Failure.mov

Troubleshooting:

- Restarting LR

- Updated to latest version of LR (8.2 build 1204643)

- Reset HDR image (to remove all adjustments)

- Tried regular DNG (non-HDR). Works as expected

- Tried 3 bracket instead of 5 bracket HDR merge. Same issue

[SeanMcCormack]

It's working for me, but I usually have a feathered edge as where the centre part touches, that's where the effect is applied. I've tried it on both a HDR.dng and one of the source files and they're both working equally as badly (sorry, never been a fan of it!).

[digitaloxygen]

Auto Mask definitely isn't a great tool but in certain situations it works well enough to not have to jump over to PS and back. The hard edge in the video was mostly to showcase the issue better. It was less obvious with a soft edge.

I have a feeling it will be a very specific set of circumstances that cause this. All the "fun" issues are like that. In my searched I found similar issues from years ago that supposedly only applied to 32-bit .tiff's or something so this might take a bit to figure out. I will be contacting Adobe about it directly. Definitely seems like a bug.

Some other interesting findings.

- Older HDR images where Auto Mask was previously used and worked as expected how exhibit the issue too.

- I think it's behaving as if Luminance Range Mask is on even though it is disabled. I even tried toggling it on / off again with no luck.

[GoldingD]

First off, turn on the mask vusability to see it better.

Second off, it may be more advantageous to turn it off, overspray your mask, then turn the range mask on, select color, click on the eye dropper then sample what you want in the mask.

[digitaloxygen]

First, this was just quick demo. Didn't think of turning on the mask overlay until after but yes it would have shown it a little better.

Second, again because it's just a demo it doesn't represent all (or even my main) use case for Auto Mask. Yes, in that demo case I could definitely use the color based Range Mask but I still need to use Auto Mask in other scenarios and need to figure out why it isn't working.

[GoldingD]

Ok, had to go to a proper computer and a proper monitor and look again (probably more the look again) and I see that on the HDR the cabinets are getting included in the mask whereas on the non HDR they are not.

And yes, I can replicate on my copy.

[digitaloxygen]

When you say " I can replicate on my copy." do you mean you are experiencing the same behaviour with the Auto Mask not working on HDR images?

[GoldingD]

digitaloxygen  wrote

When you say " I can replicate on my copy." do you mean you are experiencing the same behaviour with the Auto Mask not working on HDR images?

Yes.

Now not in a PANO, and not in at least one HDR-PANO

[digitaloxygen]

Excellent, thanks for the confirmation.

I just tried a Pano and an HDR-Pano. The standard pano worked fine but the HDR-Pano exhibited the same issue, Auto Mask didn't work.

I wondered earlier if it was because of another adjustment or correction but even doing a new HDR merge and leaving everything on defaults (i.e. making no other adjustments after the merge) I still have the problem.

Contacted support earlier today and they tried resetting all LR preferences but it did not resolve the issue so they escalated it.

[digitaloxygen]

One differentiator may be that, at least for me, the Auto Mask is behaving like the Luminance Range Mask. So sometimes it appears it's working but only if the area under the crosshair is a significantly different luminance than the neighbouring area. Maybe check for that. Came across that while showing the issue to support.

[digitaloxygen]

@davidg36166309 were you able to confirm if the "working" images were just doing Luminance Range Mask? That's what I'm seeing.

Recorded another video to showcase it. You can see in the second part where I try auto mask on the HDR image it appears to work around the window but it's doing based on luminance not color and whatever other factors the regular Auto Mask feature works on because it bleeds all over the cupboards. Very different results than the original raw file. Support claimed it was working for them but based on the sample file they sent back it wasn't they were just doing it around a window.

https://assets.adobe.com/files/Support%20Case%20ADB-5883124-G4D1/LR%20Classic%20CC%20Auto%20Mask%20H...

[Todd Shaner]

The video link won't open (spinning wheel). I suggest uploading the HDR DNG file to Dropbox or other file sharing site with your settings saved using CTRL/CMD +S.

[digitaloxygen]

All related files are here:

https://www.dropbox.com/sh/vwlf1afr6jh4mtm/AAAWPWGek3rBfecE4o_2V20ma?dl=0

[Todd Shaner]

Looking at the lowest exposure BV_20190214_9958.CR2 file and HDR DNG the raw data highlights aren't clipped, but there isn't a lot of detail in the windows. Also when you paint n the windows using Auto Mask keep the Brush Size small and Feather low (0-15). Even then you'll get spill-over in areas that have similar luminance and color to the window frame. Use the ALT key Subtract to remove the spillage. There's nothing wrong here except the window frames are very close in luminance and color to the image in the windows. The Luminace and Color Range Mask don't work well here either for the same reason.

Here's what the raw data looks like now in RawDigger with you're current bracketed files. You probably need to reshoot the bracket with a couple of lower exposure brackets, which may improve the detail in the windows.

[digitaloxygen]

Appreciate the in-depth look, though most of it isn't relevant to the issue. I have already masked and delivered the shoot with techniques similar to what you mentioned. The issue was discovered when editing these images and they are about as good as others to highlight the issue.

As per the videos (see Dropbox - LR Classic CC Auto Mask HDR Failure 2.mov for a better example) Auto Mask is clearly behaving very differently (seemingly not working) with HDR vs raw / jpeg images. Looking at the cupboards when brushing the HDR image you can see the brush spills all over them when the crosshair is on the blue walls. In raw or jpeg photos it masks around the cupboards and the brush is (mostly) only applied to the walls.

Had a lengthy call with support yesterday and they have confirmed the behaviour is odd and have (re?)escalated to senior support.  I don't use Auto Mask very often but when I do I'd like it to work as expected.

[Todd Shaner]

OK, after looking at the video and exporting the HDR DNG file to TIFF file format I can confirm what you're seeing. With the TIFF file sampling on the blue wall around the window using the Adjustment Brush with Auto Mask works well to confine the mask.

I was confusing that you were trying to work on the windows. It's still not perfect using the TIFF export file, but much better than when working on the HDR DNG file. You can use this as a workaround for now. Apply all of your Global Toning edits to the HDR DNG file and then export to TIFF or JPEG file format and then apply the Local Adjustment Brush edits.

I suggest posting this issue as a 'Problem' report at the Photoshop Family forum. Place a link to this post in the report and a link to that report in a reply here. This will help others who land here with the same issue to add their comments "Me To' vote and 'Follow' at the problem report. Thank you!

Photoshop Family Customer Community

[digitaloxygen]

Thanks for the confirmation. Good suggestion on the work around. I have indeed use similar processes in the past and it can work well in the interim.

I'll see if I can get the problem report together if I have time.

[Todd Shaner]

You can Copy & Paste what you have here to create the report. A reference link to this thread can be inserted with the words "for more details and examples." I would imagine you'll get comments from other users and hopefully Adobe Staff.

[digitaloxygen · Accepted solution]

Update: Marking this as "Answered" for now since it has been migrated over to the Feedback site and recognized as a bug by Adobe. In theory they should provide an update there once it's fixed.

Adjustment brush Auto Mask not working on HDR Images | Photoshop Family Customer Community

[digitaloxygen]

Wow is Adobe's support process ever a mess.

Got an email from someone (presumably senior support as it was escalated to them) asking me additional questions but the footer says "Note: This email address cannot receive messages. Please do not reply to this email. For a list of our support phone numbers, please visit our Contact page. " silly being it's 2019 but oh well...

Additionally it says "To review your case history or reopen the case, log in to your Adobe ID Account page." so I go into my account and find the case. It's completely empty. Zero notes from the original chat agent, senior support, etc. I add notes there anyway since they claim replying to the email won't work.

A few days later I get another email but it's in FRENCH for some insane reason. Translation says basically "we haven't heard form you". So I contact support again. They send a second copy of the last email (but in English) and tell me to reply to it (despite the footer saying not to because it cannot receive messages). We wait a bit and of course he says he didn't get the reply but he will put in a note for them to contact me.

Fingers crossed I guess. What a gong show.