It seems the monkeys have been at the file formats again...!
Open an exr with an alpha in CS2 and the image displays normally and the alpha is retained.
Open an exr with an alpha in CS3 and the alpha channel is applied to the transparency and then lost... which is really STUPID considering you might apply 0 alpha values to parts of the image you retain visually, as you might just want to use the alpha to drive an effect and not just be myopic and think it's just for transparency.
So, can this be fixed? I can't see any info on it?
Will CS2 non intel plugin work on an intel system in CS3
If not, effectively PS is useless for exr work for us.
Or is this fixed in CS4?
Tommy L2, Oct 15, 2009 7:24 AM
Yes I do Chris, but alpha isn't transparency. It's an alpha channel. We're importing files with alpha channels, not transparency as generated in PS.
PS might use alpha channels to generate transparency, but it's a bit like saying an RGBA Jpeg should behave the same, yet we know it doesn't. We all know that tga was changed, then changed back again... it's the same thing.
The problem is that EXR is the only format we can convert to from our 3d images, and we like to render in one pass and use the alpha for an an alpha, not transparency. By losing this we now have to render in two passes, one for the RGB data, and another for the alpha, then recombine them afterwards.
CS3 can support both transparency and an alpha in 32 bit at the same time. So can we not have the choice before a file is shredded? At least if we have the alpha we can choose wether to apply it as transparency, or actually use it as a selection to drive an effect. The are a whole host of passes we use alphas for, only one is transparency.
Now it means we have to keep a legacy system around to open EXR's correctly, or render 2 passes. Which is a bit of a bind to say the least considering the size of the renders. Last night it was just the one 50k render. But it took ages to process the 6gb of data to where we need it because we had to chuck it around on old gear.
How much of a hoop is it to add that UI, and how much of a hope that it will happen?
This is a huge deal for us (Pixar) as well. We are trying to integrate (finally) OpneEXR into our pipeline and when I load OpenEXR files with alpha computed by the renderman, it gets lost in Photoshop CS3/CS4.
Now the option here is to either ask Adobe to fix this in CS4 and allow an option at the loader level or we have to write our own OpenEXR reader.
To better understand since I don't get the difference between transparency and alpha (for us, we just deal with alpha) how the OpenERX file should be structured so that we can retain the alpha/transparency from the file ? Maybe we can modify the renderman driver.
Thanks Guido, it helps to know it's not just us little chaps having trouble.
Perhaps you can split your alphas off before PS get's to it?..
I'm not sure we can with our renderer... we might have to get it to render RGB, then again for the alpha. For standard alphas that's a hoop jump, but if we wan't shadow matte in the alpha for example, then thats a big pain as it effectively means we'll be getting close to doubling our rendering times... and we normally work on 8k+ renders, so that's a bit of a nause.
with OpenEXR it is standard practice to interpret the "A"
channel as pre-multiplied alpha. Compositing a foreground
layer over a background layer is correctly done like this
(see http://www.openexr.com/TechnicalIntroduction.pdf, page 5):
composite = foreground + (1-alpha) * background
Programs that read and write OpenEXR files generally adhere
to this convention. I don't know how transparency works in
Photoshop; your message suggests that PS transparency is not
the same as alpha. This would imply that PS should not treat
OpenEXR's alpha channel as if it was transparency. Of course,
PS could add transparency channels as needed when it writes
Florian - you are confusing the terms alpha (any extra channel) with transparency (what you called alpha). Photoshop reads and writes OpenEXR files exactly as you stated - with premultiplied transparency.
Apparently some users thought EXR supported general alpha channels and are now confused when those channels are treated as transparency (as the file format specifies).
Alpha Channel - any extra channel beyond the color channels. This has nothing to do with transparency (except as a historical footnote). Anything can be stored in an alpha channel, it does not have to be related to the color channels at all. You can have many alpha channels in an image.
Transparency - a subset of extra channels, closely related to the color channels, with special meaning. Can also be called Opacity (transparency and opacity are the reverse of one another). You can only have one transparency channel in an image. This may be premultiplied with the color data, depending on the file format specification.
Guido - Photoshop DOES preserve the transparency/opacity data from EXR files, exactly as specified by the file format spec. The only exception was in CS2 when Photoshop did not support 32 bit transparency, in CS2 we just read the transparency channel in as an alpha channel.
If you are trying to use the EXR "A" channel for something other than transparency - STOP. The file format does not allow that. You can add other named channels in EXR, but Photoshop doesn't yet support them.
Chris, I think your wrong on this. I think that's backed up by TD's from Pixar and the originator from ILM posting as such. It's being interpreted in the wrong manner, ie assuming transparency = alpha.
Exr doesn't have transparency in it's spec. Its RGB+A. But PS is presuming that A = Transparency. Not only that, it's applying it and deleting it before the file is viewed, which means when it's used in pre-multiplied it's actually taking away image data that can't be put back.
It's not done with any other format, and it's not done with tga now it's been changed back. It's the very same thing.
PS is making irreversible changes to the document when it's opened. It doesn't matter even if later you define the alpha as the transparency, it's nuking the image that's the problem. Pre-multiplied means that for any pixel value except 0 in the alpha, the alpha will remove some of the image + the background it's "transparent" against.
If I have an alpha, I can choose to apply it to the image to make transparency as I wish.
It's impossible to rebuild an Exr back to what it was once PS has got hold of it, because by applying the alpha as transparency it has deleted RGB image data.
A 1/2 way house would be to have it as a layer mask... at least then both your transparency interpretation and the image data is retained and the user can have the choice.
Progress - read Florian's statement and the EXR spec. again. His "A" is defined as transparency. The explanation is just confused by referring to it as alpha (which could be any additional channel, not just transparency). But the math, and the requirement for premultiplication means it must be transparency. That is the only interpretation available, and the interpretation that Photoshop uses.
Yes, premultiplied transparency does lose image data -- but that is the way the EXR format is defined. Your "nuking" happened when you saved the EXR file, because it has to be premultiplied (transparent areas go to zero in the color channels).
The only change Photoshop makes is to un-multiply the data, because Photoshop does not work with premultiplied data. The data is re-multiplied when you save the file, resulting in no net change to the file data. Transparent EXR files round trip just fine through Photoshop, as far as I can test with files from ILM, or those that I have created.
If you wish to have an arbitrary alpha channel (one not treated as transparency) in the EXR format, you will need a third party plugin. EXR does allow for extra channels, but the Photoshop EXR plugin does not support them yet.
Chris, and Progress,
I guess I don't understand what Photoshop does with OpenEXR's A channel.
When you open an EXR image with RGB and A channels in PS, and then
immediately save the image, will the new file not be identical to the original?
The color channels may be altered slightly due to un-multiplying and re-multiplying (remember: Photoshop does not use premultiplication, so we have to un-multiply to use the image). So, the least significant bits may change.
In an EXR image it is legal to have pixels with non-zero RGB values
and a zero or nearly zero A value. Such a pixel represents an object
that emits light but is otherwise transparent, for example, a candle
flame. (If you place a candle between a bright light source and a
screen, the flame casts no shadow.) In this case un-premultiplying
leads to infinite or very large RGB values.
Chris, I agree with you, the A channel should be used for alpha (in
the transparency sense). I don't see how PS could possibly guess that
A is in some files really a depth channel. That's why EXR supports
arbitrary channels, and why the specification recommends using Z for
depth. On the other hand, this would be less of an issue if PS
guaranteed that RGBA data pass through unaltered unless the user
explicitly modifies the image.
Pre-multiplying is done at the render. It's not something the Exr 'does'.
You cannot un-multiply a render... you have to render it in a different manner completely. It is impossible to un-multiply the image because a pre-multiplied image always contains background data as a % related to the alpha strength. The A is not a by-product of the render's opacity, it is merely a single channel representation.
Photoshop does not just alter "slightly" the files. In your example, if you have a very bright but transparent object, the effect that PS has is to almost entirely remove that image data and the alpha. You cannot get that image data back.
Florian, what PS now does is to take the A and use those values as to generate a selection, and delete from the RGB channels. So if you A value is black, areas are entirely deleted, if the value is 50%, 50% is deleted and so forth. And it does this automatically, and so the image is destroyed, and the alpha is lost.
i On the other hand, this would be less of an issue if PS
guaranteed that RGBA data pass through unaltered unless the user
explicitly modifies the image.
If people are having trouble seeing what the problem is, let me know and i'll post some examples tonight.
Progress - yes, you can un-multiply (aka divide with appropriate handling for zeros)! You have to in order to bring a premultiplied image into an app that doesn't work with premultiplied data. We do it all the time with file format that specify premultiplied data (like TIFF). Premultiplication does not necessarily happen at the render stage - it may, depending on the renderer, but in many cases it is an explicit step since some file formats want premultiplied data and others do not.
And your description of what Photoshop does with the transparency channel of EXR files is very, very wrong. The transparency is not lost in any way (assuming you are using CS3 or CS4 extended, since the standard version does not support 32 bit transparency/layers/compositing).
Photoshop cannot "pass through unaltered" because the file format is premultiplied and Photoshop doesn't work with premultiplied data. We have to undo the multiplication. Then the file format requires that it be multiplied before saving.
And it looks like Guido's problem was some people using extended, and other people using standard (who could not see the transparency correctly, because standard does not support 32 bit/channel transparency).
What Exr is doing is applying the A into the image and not making it editable...althought the transparency is applied, the transparency is effectively "lost" because it cannot be got at.
Another point that you might not be aware of is that the RGB data is built from the RGB of the object against the background. The background might often contributes nothing to the alpha, but does to the RGB... so when the alpha removes the area it removes RGB data. Obviously this is exagerated when using pre-multiplied values of not 0 or 255, because it removes a proportion of the RGB (including the background) related to the A value.
Because of the way PS can't handle pm, it is the exact reason why it shouldn't assume alpha isn't transparency.
If I render an object with no transparency with the values 128,128,128 and A=0 then render it with 50% opacity, the image is 128,128,128 and A=128. The problem is that although the pixel values are stil 128 in the RGB, the image is transparent! You can't get that "solid" 128 RGB data out because PS ties it's transparency in with it... you can't unhook the two apart from each other. There's no layer mask to lose, or alpha, it's 'baked' the transparency into the image...
It's not so much that the transparency is lost, it's that the image is irrevocably broken... I can re-generate an alpha from the transparency... i can't see how i get those pixels back to full opacity though.
perhaps there is a way of doing it... how do I get a that image back solid? The pixel values haven't changed, they're still 128, 128,128... the alpha is gone, there is no layer mask, the layer is 100% opaque... yet there is a transparency value attached to each pixel that I can't get at.
The only way I know how is to put the exact colour in the background that was used in the renderer... but it might not be a colour used as the background, it might be an image... it might be a background plate which normally carry no alpha contribution... that's the problem.
You don't arbitrarily apply transparency in 8 or 16 bit to other formats before they are opened... why do it in 32 all of a sudden? They all support transparency once in PS (well its PS that supports it, not the format)... same as Exr.
In fact, it was this exact problem that prompted the backstep on alpha. I can't see why you can't see this as the same thing, because from our end it is.
If you want Chris, I can send you the exact files. A simple transparent box floating over a background, rendered. Normal alpha generated from the opacity straight from the file. All standard stuff, straight from a standard renderer. Your challenge is to tell me what was in the background. Your 2nd one will be to re-create it. With the CS2 you can... with CS3 you can't.
But does this mean we can load the CS3 non extended plugin in, or does the app do the damage? Could that be perhaps made available if it solves the problem?
Progress - no, it is not broken, it is working exactly as intended, exactly as the file format spec. says it should.
In the openEXR file format, the "A" channel is defined as transparency/opacity, and the color data must be premultiplied with that transparency/opacity channel.
If you render an image with transparency/opacity into OpenEXR, Photoshop will open that image with transparency/opacity - exactly as the file format spec. says it should.
Many file formats include transparency/opacity and are premultiplied (like TIFF), and we have been handling them exactly the same way since Photoshop 3.0. There has been no change - we have to treat transparency/opacity as transparency/opacity.
With CS2, it was broken - because 32 bit/channel did not support transparency or layers (they are connected). So, in CS2 (and Photoshop standard) you get the same result as if you'd opened a transparent TIFF back in Photoshop 2.5 -- the color data, with transparency stored in an alpha channel (because we had no way to display or use the transparency).
Your description is very confused, so I'm not really sure what you are trying to accomplish. But the OpenEXR support is working exactly as it must work.
If I open a tiff with a premultiplied alpha it stays as RGB and A. No transparency is seen.
If I open a targa with a premultiplied alpha it stays as RGB and A. No transparency is seen.
If I open a jpeg with a premultiplied alpha it stays as RGB and A. No transparency is seen.
Yet if I open an EXR with a premultiplied alpha it removes A and applies it to the RGB. It doesn't alter the RGB values but it adds a transparency value that PS understands but is not editable.
Thus with PS's handling of transparency, I now end up with transparency embedded into the file which cannot be edited.
It must work another way because CS3 Standard and CS2 previous work another way. Just because 32 bit in CS3e supports transparency does not mean the A should be applied to the image to produce it. Just like it doesn't for every other format in 8 & 16 bit, which also support transparency.
I'll make it clear... PS deletes RGB image data based on alpha values. That is not correct. No other format does this. And this is why Targa had to be changed back. Because it was an incorrect interpretation of what an alpha is. An alpha is not transparency, it's a greyscale representation of transparency. There is nothing in the EXR or other format that says it has to be applied to the RGB image, and since other formats dont behave the same way I don't see why Exr should either.
It's so VERY VERY broken. Do you think I would bother posting here if there was no effect on the image?
Or am I missing something fundamental that allows me to retrieve RGB data from the file which has been made transparent, even though there is no layer mask or alpha?
Is the plugin different on CS3 standard? Or is it the code in the extended application that adopts a different interpretation.
If you open a TIFF with transparency, Photoshop shows transparency - otherwise the TIFF file was written incorrectly (something certain 3D apps mess up from time to time) or you wrote the extra channel as an arbitrary alpha channel. TIFF does support transparency/opacity and arbitrary alpha channels.
TGA should support transparency, but we backed off because so many people were misusing the file format. We still have people who want to open it one way and save it another (meaning we should prompt for how to handle the fourth channel at open and save).
JPEG does not support transparency at all.
The transparency/opacity in an EXR file is not lost - it's right there as transparency/opacity. Where it is supposed to be. Nothing is removed. Nothing is deleted. Oother file formats that support transparency/opacity work the same way. Other file formats that are premultiplied work exactly the same way.
CS2 was just broken with regard to 32 bit transparency. CS3 standard tries to follow the CS2 path because it also does not support 32 bit transparency. But CS3 (and CS4) extended are working exactly as they must work.
Look: transparency/opacity is transparency/opacity and must stay transparency/opacity. If you want to use that extra channel for something other than transparency/opacity - use another file format that supports that concept. Right now you are either hopelessly confused about the concept, or just using the wrong file format to do what you are trying to do.
I will post examples if i have time this week, if not next.
Chris, if you open a tiff with an alpha, it does not display transparency. It displays RGB and an Alpha. Even if you save it with an alpha, it opens back up with an alpha.
However, I can introduce transparency and save it as that as well as the alpha.
So what is different then? If alpha must equal transparency, why doesn't tiff behave the same. Moreover, why can we have a transparency and a different alpha in the same document, yet the two are apparently the same? That does not make sence in relation to your point of view.
Targa, as Adobe found out, was changed to be wrong. Adobe interpreted it in one way, and the rest of the digital world told Adobe why it was wrong. This is largely the same. It's broken/wrong.
alpha != transparency -- they are not the same and the terms have not meant the same thing in over 20 years.
Progress - remember, alpha can have multiple meanings (arbitrary alpha channel, or transparency/opacity). Please be clear about which meaning you are using. If you mean transparency or opacity, say so. If you mean some extra channel that has no relation to the color channels (alpha channel), say so.
TIFF can contain transparency/opacity AND/OR arbitrary alpha channels. If you have a TIFF with transparency/opacity, then the TIFF will open with transparency. Some 3D packages have bugs in their TIFF support where they fail to mark the transparency channel as transparency, so it opens as an alpha channel instead.
OpenEXR does not support arbitrary alpha channels, only transparency/opacity.
LOL! TGA was changed to follow the spec. Some users were using it incorrectly. So we changed it back to NOT following the spec. It's still maybe 60/40 on the usage of TGA, and we get complaints from both sides.
OpenEXR is implemented correctly in Photoshop, exactly as the spec. says it has to be implemented.
I loaded the Blobbies.exr file from the OpenEXR sample image collection
into Photoshop CS3. The file has an A channel (the kind that PS calls
"transparency" and most other people call "alpha").
I used the paintbrush tool to paint onto both transparent and opaque
areas of the image, and I used the eraser tool to create holes in the
opaque areas. Then I saved the result in another EXR file and looked
at the file's R, G, B and A channels with exrdisplay. The data in the
file were what I expected: the paintbrush had moved the A values closer
to 1.0 and the eraser had moved the A values closer to 0.0. The RGB
values seemed to have been properly premultiplied with A. Compositing
the painted-on image on top of another image in PS also produced the
As far as I can tell Chris is right, PS handles OpenEXR's A channel
correctly. What am I missing?
The case where A is zero and R, G and B are not zero means something
inherently different in PS and EXR. PS could probably do a pretty good
approximation of the EXR semantics by clamping EXR's A to the range
[1e-9, 1] before un-premultiplying. For all practical purposes an A
value of 1e-9 is zero, but it would allow PS to preserve non-zero
RGB values and to un-un-premultiply on save.
"As far as I can tell Chris is right, PS handles OpenEXR's A channel
correctly. What am I missing? "
Depends on what you open it back up in. Open it after saving out in PS Extended again and you'll see that the A channel has gone, and the opaque areas are now holes (perhaps how you left it) So you can no longer edit the A channel except destructively, by adding even more transparency. You cannot reduce the transparency.
After saving the image with holes, open it back up again in PS and try and paint back in opacity on the A channel, without affecting RGB at all. I could do this before because I had direct access to the alpha. With the alpha seperated as before I can adjust the A without affecting the image's RGB. Not only that, I don't lose areas set to black on the alpha from RGB when it opened. I now do. They are removed, as per holes.
Before, if i received an image with a background that would be removed by alpha when applied, I didn't need to know what that background was... now I do because it's removed before I can see it.
I will post links later this week hopefully when I have time.
There is a fundamental difference between the two behaviours. The latter removes functionality in a very big way.
Progress: I so agree with you. I sit on OpenEXR images that are currently useless right now. I don´t care if the alpha channel is transparency or not - I just want the possibility to edit that channel. Now I can´t!
OpenEXR is the right file format for me as it supports high dynamic range and is (was) great for renderings. I could choose TIFF too but it´s the same problem there. The alpha is applied and the channel removed. My previous workflow was smooth when I could include the alpha directly into the file instead of render two files for every image.
It worked in the previous Photoshop versions so why change the plugin now?
Yes, if you specify transparency/opacity in the file, Photoshop will open that file with transparency/opacity. That's what it is supposed to do. The channel is not removed or applied - it is right there in front of you!
The OpenEXR plugin was changed in CS3 because CS3 was the first version to support 32 bit/channel transparency. Photoshop CS2 supported OpenEXR and 32 bit images, but not transparency. That means that OpenEXR support was broken in CS2, and was fixed with CS3.
In short - it did not work in previous versions, and your problem (wanting to edit transparency data directly) has little to do with the file format. You relied on a bug (or lack of a feature) to do something unnatural for that file format.
Chris, I guess we have established that Photoshop does
The Right Thing (TM) when opening an OpenEXR file with
an A channel or an equivalent TIFF file. On the other
hand, the request for an option to open the EXR A channel
as a PS alpha channel for independent editing sounds
entirely reasonable. I guess progress is requesting a
Progress, if you use the OpenEXR A channel to store
information other than how transparent a pixel is, then
how is software such as Photoshop supposed to know when
the A channel really is an A channel and when it is, say,
depth by another name? If you are storing depth in the
file, you should probably call the depth channel Z as the
EXR documentation recommends.
You can ask Renderman to split the output of a single
render pass into multiple files if packing all the output
channels into a single file is inconvenient. You can,
for example, have Renderman write two EXR files, one
with the R, G and B channels and one that contains only Z.
This way you can edit the RGB image in Photoshop without
altering or even loading the Z channel.
I've probably complicated the conversation by the suggestion that A be used for something else. The ability to use A as non transparent information is a bonus, but even the ability either edit A or simply for A not to be directtly translated into transparency would be great.
The simple problem is that elements such as background info might be specified quite normally as black in the alpha, then these are lost when opened. PS is deleting data created. I know it's only applying transparency, but it's not editable. I can't get to the data before PS deletes it.
PS treats data that is 100% transparent as 100% redundant. It isn't. I might want to get to that data.
Could it be done so it opens as a layer mask instead? Then both camps would be happy then.
(I guess there are work arounds, but I can't help thinking there's no good reason that A should automatically be converted into transparency)
Progress - The extended and standard behavior is just a switch in the code, not different code. It just does slightly different assignments based on the host licensing.
And again-- Photoshop is not deleting any data. Your only problem here is that you want to edit the transparency/opacity data directly, and Photoshop does not let you do that easily. Opening the transparency as a layer mask adds some serious problems with roundtripping and compatibility with other file formats (because most of them do not support layer masks).
"A" in OpenEXR _is_ transparency. That is the way it is defined, that is what it must be. Using that channel for anything else breaks workflows (which is why I hate putting it in an alpha channel in standard).
If the last post isn't sarcastic, then that would be a very useful solution to a number of scenarios where transparency is embedded and prevents RGB editing of data hidden by a transparency which is not editable.
I still maintain that A isn't transparency, but a representation of transparency... but i'm willing to drop the pedantry if it gets us somewhere.