gamut warning/selection discrepancy

[marks60810224]

I have my gamut warning color set to gray (this is not essential to the problem; I mention it only to make it easier to describe my gamut warning color).

Two issues:

(1) When I use View>Gamut Warning, all the out of gamut pixels are grayed out, as they should be. However, when I use Select>Color Range>Out of Gamut, the selected pixels are not exactly all and only the grayed out pixels.

In other words, View>Gamut Warning and Select>Color Range>Out of Gamut do not agree. They might agree on some pixels but not others.

(2) When I use View>Gamut Warning, usually I turn the grayed out pixels to in gamut by desaturating them or painting over them. However, sometimes this causes nearby pixels that were NOT grayed out to become grayed out as if they are now out of gamut. So what I don't understand is how desaturating an IN gamut pixel could cause it to go out of gamut, or how painting over an IN gamut pixel with an IN gamut color could cause the pixel to go out of gamut.

[Barbara Ash]

Photoshop displays a selection around pixels that are 50% or more selected. If the out of gamut colors are borderline out of gamut, they may not appear to be selected. It is also possible that some averaging is going on.

Aside from the technical issues, it seems that mainly you want to get all the colors in gamut. You can do that by choosing Image > Mode > CMYK. As long as you're working in CMYK all colors will be in gamut. If you prefer to work in RGB, you can choose Image > Mode > RGB to switch back again.

[D Fosse]

From what I can see, Gamut Warning just has a higher threshold than the other indicators - IOW smaller areas are indicated.

Personally I have always found these overlays of zero value, simply because they don't tell you how much out of gamut. A visual soft proof (ctrl Y) gives you much more realistic feedback. This is where a wide gamut monitor is useful. It should cover all CMYK profiles and most RGB inkjet profiles completely, so that you actually see the clipping. It's very hard to make reliable judgements when you can't see what's going on.

A wholesale desaturation of all out-of-gamut areas, regardless of how much, is a very heavy-handed way to do it. You should also consult the histogram, and view individual channels, for a more targeted approach. Some even argue that you should just leave it to the profile - just pick the rendering intent that works best visually. I wouldn't go that far, but often small areas can be allowed to clip without any ill effects.

[marks60810224]

Thanks Barbara and D for your help.

But I don't understand, which is my fault because I'm not used to thinking in the framework you mention.

I'll state my workflow (in detail for sake of exactness) and how I understand it, then maybe we can figure out more from there:

I shoot RAW Adobe RGB. I open the RAW in Photoshop as Adobe RGB (1998), 16 bit, 300 ppi.

I have two intended outputs: (1) prints (on a certain paper) and (2) web.

I have an NEC wide gamut monitor and SpectraView.

PRINTS

I created a SpectraView profile so that (as best I could devise) what I see in Photoshop matches the prints. I have the monitor set to that SpectraView profile.

In Photoshop, View>Proof Setup>Custom with: 'Device to Simulate' set to the paper/printer profile I downloaded from the paper manufacturer. 'Rendering Intent' set to Perceptual. Checked for 'Black Point Compensation'.

Then I work on the image. I do quite a bit, but usually the most important part is overall increasing brightness and overall desaturating (in other words, if I printed the image with no adjustment, the print would look too dark and saturated). And one of the things I do when working on the image is to check for out of gamut by View>Gamut Warning. My understanding is that this should mark all pixels that are out of gamut relative to the View>Proof Setup>Custom settings I've made (in other words, all the pixels that are out of gamut for the printer and paper I'll be printing with). 

I usually don't worry about a few scattered out of gamut pixels (I don't think the human eye would see them in the print), but for larger areas, I use various techniques to bring them into gamut. Usually, I gradually test more and more desaturation until right around the point the pixels pop from gray back to color. Also, for the pixels that have popped back to color, I can see what that color is, so I can tell whether the desaturation has resulted in too great of harm to the image.

However, also I would like to take advantage of Select>Color Range>Out Of Gamut, so that when I use the desaturation brush, it doesn't cover any in gamut pixels. This is why I've asked why Select>Color Range>Out Of Gamut doesn't match View>Gamut Warning. And usually, the selection is greater than just the gray alert pixels. So Select>Color Range>Out Of Gamut is saying there is more in the photo that is out of gamut than View>Gamut Warning says. This seems to be the opposite of "If the out of gamut colors are borderline out of gamut, they may not appear to be selected." Also, I don't understand pixels being only partially selected (isn't a pixel either in a selection or not?), as indeed I've never understood feathering values that include fractions of pixels.

Then I save to TIF.

In the Photoshop Print Settings box I have: 'Color Handling' set to Photoshop Manages Color. 'Printer Profile' set to the paper profile I downloaded from the paper manufacturer. Normal Printing. 'Rendering Intent' set to Perceptual. Checked for 'Black Point Compensation'. And Print Settings>Main has 'Color Intensity' set to Manual, and 'Color Correction' set to None.

This results in prints that pretty much match what I see in Photoshop (though I'd like to improve the match even more).

WEB

With SpectraView, I switch the monitor to the SpectraView preset profile called 'SRGB Emulation'.

In Photoshop, I turn off View>Proof Colors (or, it looks exactly the same with View>Proof Setup set to Internet Standard RGB (sRGB)).

I open the TIF I made for print, set Convert To Profile>Profile to sRGB IEC61966-2.1. Then usually the image doesn't need much work other than overall decreasing brightness and overall more saturation (i.e. reversing what I did for the print version. And then checking for out of gamut, mutatis mutandis, as with the print. Then Save As to JPEG.

[D Fosse]

You seem to be all set - your workflow is good, and since you have a wide gamut monitor you can take full advantage of visual soft proof to your printer profile. I would strongly advise that over the gamut overlays.

Just one little thing:

if I printed the image with no adjustment, the print would look too dark

That indicates your monitor white point is set too high. Monitor white should be a close visual match to paper white. Since this depends on ambient light, application interface and print viewing light, there are no "correct" values here - just get them to match visually and nevermind the numbers.

Do the same for black point. It should match max ink density for the paper. Good glossy inkjet papers rarely exceed a contrast ratio of 250:1 or at best 300:1. With a white point at, say, 120 cd/m2, that gives you a black point at 0.4 or 0.45 cd/m2.

This is the most important thing you can do to match screen and print, so that what you see is what you get.

---

As for gamut clipping, the most objectionable effect is a dense, opaque effect where texture and detail is lost. It lacks "air". Clipping may not necessarily be a problem. The soft proof will tell you if it is.

[marks60810224]

The reason I'd like to be able to better trust Photoshop's out of gamut indications, is that it would save me ink/paper cost when printing. I'd rather not press the print button when some of the areas of the prints are destined to not look right because of gamut.

I first became aware of the need to watch out for gamut when I printed an image that, in Photoshop, has a blue background but the background was purple in the print - radically different from what I saw in Photoshop, and then, sure enough, I discovered that it was an out of gamut area.

/

I'm wondering whether your thing about white point and black point is subsumed by my process with monitor targets (these display targets set brightness, color balance, gamma, contrast ratio, and color space).

This next part is partially redundant with my earlier post, only because I want to keep the context of the process clear:

Aside from gamut, what I meant about prints being dark and saturated, is not that the prints don't match what I see in Photoshop. Prints do match what I see in Photoshop. And these images are correctly exposed in the camera (more exposure would make them clip in the highlights). In other words, suppose I printed the image without adjusting it in Photoshop, then the print would be too dark and saturated. Okay, so maybe the printer is at fault (putting down too much ink)? I'm not sure, but my guess is it is not a printer fault, based on this reasoning: SpectraView has a preset target called 'Print Standard'. If I set the monitor to that target, then the print very closely matches what I see in Photoshop. And I take it that the Print Standard preset does show basically what printers in general do print. And my own custom target for the particular printer/paper is close to Print Standard too.

So, to recap, I set the monitor to my custom SpectraView target (which somewhat darkens and saturates the display, relative to, say the SpectraView preset target called 'sRGB Emulation'). Then in Photoshop, I set the Proof Setup to my printer/paper. At this point, the image in Photoshop is too dark and saturated, and if I printed it, then the print would match what I see in Photoshop. So, that's good in the first sense: whatever I print matches whatever I see in Photoshop. So next, I increase brightness and decrease saturation in Photoshop and save as my print TIF, and then I print, and the print looks good and it matches what I see in Photoshop. So I think this logic and process is all correct?

Then for the JPG for the web, I switch the monitor to sRGB Emulation (which makes the display brighter and more saturated), set the Proof Setup to sRGB, convert the print TIF to sRGB, then decrease brightness and increase saturation (to reverse what I did to make the print TIF), then save as JPG.

But I'm wondering whether I should reverse that procedure - instead make the web image first (in sRGB Emulation and JPEG Proof Setup), save as both TIF (to later work on for prints) and JPG (for the web); and  this way, I wouldn't have to decrease brightness and increase saturation for the JPG. Then, switch to my custom print target and printer/paper Proof Setup, then increase brightness and decrease saturation and save as TIF for printing. In other words, instead of having to adjust brightness/saturation for print and then readjust it back for JPG, I would only have to adjust it once when making the TIF, so that the image would not be threatened by the degradation of adjusting it then adjusting it back again. (On the other hand, in my informal testing, I found that there is no observable degradation. If I brighten/desaturate and then darken/saturate back, then the image looks just like it did at the start with no adjustments.)

[Barbara Ash]

When it comes to partially selected pixels and feathering, the partially selected pixels are less than 100% opaque.

If you're not already doing so, it might be helpful for you to put the brightness and saturation adjustments on a separate adjustment layer. That way you can edit the mask on the adjustment layer non-destructively until the image looks good to you.

If you feel that Gamut Warning is more accurate than Color Range, you could also try using them in conjunction. First View > Gamut Warning. Then Color Range. That would allow you to easily select all of the gray areas. Then add a Hue/Sat adjustment layer. The gray areas will be masked. Lower the Saturation slider until the larger gray areas disappear. This might be a less tedious method than using the Sponge tool to desaturate.

[marks60810224]

Now I see that I had a serious misunderstanding of feathering. So let me try to summarize how I understand it now, and maybe that needs even further correction:

Feathering uses opaqueness. It uses a grayscale method (8 bit grayscale, so 256 possible values?). The pure white is no opaqueness, the pure black is full opaqueness, and the grays in between are degrees of partial opaqueness. The pixel number you set is the total distance (in pixels) between the farthest pixel affected by the feathering inside the selection and the farthest pixel affected by the feathering outside the selection. And then when you apply an adjustment to the feathered selection, the strength of the result is determined for pixels according to their feathering opaqueness.

But I still don't understand how fractions of pixels work. How would the distance in pixels be a fraction of a pixel? (Though, practically it doesn't matter for me, because I've never had to feather more precisely than within a difference of a pixel. So I'm just curious for sake of understanding.)

Also, is this the same basic idea in how hardness and softness of brushes work?

/

About using Gamut Warning and Color Range together, I don't understand your suggestion. When I use Gamut Warning, it gives me an overlay color (of whatever color I choose - gray in my case) on the out of gamut pixels. But that overlay color is not one that I can sample with the eyedropper to use in Color Range. If I sample an out of gamut pixel, then the sampled color is the original out of gamut color, not the gray. So I don't see a way to make Color Range select just those pixels that have the overlay color on them. Instead, Color Range has a selection option for out of gamut, but it results in a lot more pixels being selected than those that Gamut Warning shows with the overlay color.

/

Meanwhile, it's not clear to me what exactly Gamut Warning and Color Range out of gamut are supposed to be finding. I would think it's all and only those pixels that are out of gamut relative to the printer/paper profile chosen in Proof Setup (or, for example, sRGB, or monitor gamut, if either is set in Proof Setup). But if that's the case, then why would Gamut Warning and Color Range pick a different set of pixels? Also, I have a book that says "When the Gamut Warning is activated, Photoshop will place an overlay tone over any colors in the image that are out of gamut for the current CMYK setup in the Color Settings dialog box. You can also use it to display out-of-gamut colors if you have selected an inkjet profile as the current proofing space." Does "current proofing space" mean Proof Setup? And what if I have Proof Setup set to something other than an "inkjet profile" ("inkjet profile" means a paper manufacturer's printer/paper profile?) such as sRGB or other?

[D Fosse]

Gamut warning shows out-of-gamut for whatever you have as proof profile in Proof Setup. It can be a CMYK profile or an RGB profile. It defaults to your working CMYK.

But how much out of gamut? It's rather meaningless to mark a pixel as out of gamut the instant a channel hits 0 or 255. There has to be a threshold if it's supposed to be useful. But exactly where you set that threshold is a bit more arbitrary. A small difference here can mean a large difference in area.

[marks60810224]

Thanks, got it on Gamut Warning.

I totally see your point about the amount amount of difference.

Also, a curious thing to me is that sometimes if it's out of gamut then the print result is a very different color. I mentioned an image that has a rich blue background, but when printed, it came out decidedly purple. (That was unacceptable as it ruined the emotional effect of the picture.) Would that be because the blue was EXTREMELY out of gamut? Was it so out of gamut that the output device just freaks out and gives you a bizarrely different color? It's situations like that that I am most concerned about. Usually, good skin tones are not out of gamut, so that's not an issue for me. And if an object doesn't have to be an exact color to look good, then I don't mind that it might shift. For example, if some unimportant person in the picture has a sweater that prints a slightly different shade of green then that might be okay with me.

[Barbara Ash]

What most people consider a good blue is difficult to reproduce in CMYK. The CMYK blues tend when put into gamut tend to look purple, as you experienced, although they may look ok or at least not as purple on the monitor. When working with blue I always check the C and M percentages and usually adjust the percentages manually.

[marks60810224]

Thanks. Would you please tell me more about what you mean by "check the C and M percentages and usually adjust the percentages manually"? How do you do that?

[Barbara Ash]

Sorry for the delay in answering. I'm working on a project with a tight deadline.

Open the Info panel and select the Color Sampler tool. Many people like to set the Color Sampler to 3 x 3 pixels instead of 1 x 1 pixel. Click in an area of blue pixels. The Info panel will display the color value. Regardless of the image color mode, you can change the display to any mode. If an exclamation mark appears next to C, M, Y, or K, those colors are out of gamut.

Using any number of techniques, you can then adjust the color values until they are within gamut. The Info panel will update the ink percentages accordingly. The numbers on the left are the original color values and the ones on the right, after the slash, are the newly adjusted values.

I know from experience that I don't want the M to be higher than the C percentage, and preferably lower than the C percentage. Otherwise, you end up with a purplish blue. If your vendor is using Pantone inks, you can use the Pantone CMYK books to get a good idea of what the ink percentages will look like when printed. I'm referring to the Pantone books that are printed on paper. Looking at the colors on the web is, of course, not as accurate.  If your vendor is in the USA, they are probably using Pantone. Other ink manufacturers, like TruMatch and Toyo, have similar books.

[marks60810224]

Thanks, Barbara. Please, no need to apologize; totally understand deadline crunches. And you're giving me free info and advice that I very much appreciate.

I looked at the Info tab, but I only somewhat follow what you're doing; basically confused and not getting it. In general, I see now that I need to take some time to more systematically study the fundamentals of digital color, otherwise I'd be asking for too many explanations about all of this.

[D Fosse]

Wait a second. This is inkjet printing we're talking about here, right? CMYK isn't relevant. Inkjet printer profiles are RGB.

[marks60810224]

Inkjet.

[marks60810224]

But this opens up another can of worms. My immediate project is to make inkjet prints and also display on the web. But later I'll want to use the images for books made by offset printing. So I don't know whether my TIFs for inkjet will also be good for offset printing, and whether any adjustments needed will pretty much be global on each image, or whether I'd have to basically redo all the details differently for each image in Photoshop specifically to look good in offset printing.

[Barbara Ash]

My misunderstanding. I thought we were talking about both CMYK and RGB. It's much easier to get good blues in RGB.

[marks60810224]

So back to the blues: I'm wondering why my out of gamut blues printed as so dramatically purple. What I'd like to understand: What is SUPPOSED to happen with out of gamut colors that aren't corrected in Photoshop? I would think they would print as roughly the same hue but less saturated. Yet, my out of gamut blues printed as a distinctly different hue.

[D Fosse]

This blue to purple shift is a "well known secret" - but for some reason very little is written about it. I've been trying to figure it out myself.

It has something to do with color spaces not being perceptually uniform in this area - IOW a shift in values doesn't correspond to the same shift in perception. And in the deep blue area there is some non-linearity that causes this.

I'm looking for a good explanation myself. If anyone finds one, please post it here.

[D Fosse]

The problem with desaturation is that it affects all channels, at all tonalities. If the clipping is at the low end of the blue channel, there is no reason to touch the high end of the green and red channels. Then you should specifically target the low end of the blue channel and nothing else. This can be done with, say, "Blend If" and a Selectice Color or Channel Mixer layer, just to sketch out an alternative approach.

Very often you can remove clipping with no perceptible loss of overall saturation.

[marks60810224]

Yes, I can see that desaturating could be a crude method. I'll have to research more about how to use the methods you mentioned.

[Conrad C]

I'm replying to this old thread because of what I seem to have discovered after getting confused about the gamut warnings in Photoshop. (This came up because I'm updating a book.)

I was finding, as marks60810224 did in the original post, that View > Gamut Warning and Select > Color Range / Out of Gamut returned different selections. So I tried changing profiles in various places, and came to the following conclusion:

View > Gamut Warning is based on, as discussed, the profile selected in the Device to Simulate menu in the Customize Proof Condition dialog box. Change the profile, the gamut warning changes. But the gamut warning in Select > Color Range / Out of Gamut was not changing. After experimenting, it seems like Color Range / Out of Gamut corresponds to the profile selected for CMYK Working Space in the Color Settings dialog box. Which seems weird. But if true, it means we cannot use Select > Color Range / Out of Gamut to accurately select for RGB printing workflows. (I never used Color Range to edit for my Epson printer anyway, but some might have tried.)

Anyone want to try and verify that, in case I missed something? What I did was do a Color Range / Out of Gamut, then save that selection as the mask for a Solid Color layer so that I could see multiple Color Range / Out of Gamut in the Layers panel, repeat for different profile selections in Customize Proof Condition and then Color Settings, then turn the different solid color layers on and off to compare their areas.