This content has been marked as final. Show 35 replies
I don't know anything about Capture One. I have never seen it. Adobe has created the DNG file format, and treats it as somewhat of an "open" and documented format. They have provided the complete file specification. However, each company that decides to support the DNG format has the option of deciding how they implement that support. Some of these other software packages support DNG, but only from the cameras that they already support. So I suspect this is a problem with how the developers of Capture One have chosen to implement their DNG support.
I just read my answer, and I wonder if I could have found a way to use the word, "support" for a few more times. :)
Thanks, Jim. You've been very ... supportive ;)
My line of thought was this.
If I'm not mistaken, when any RAW format is converted to DNG, a kind of "camera description" is included to allow programs that doesn't "know" that camera to interpret the raw data properly.
But, there are a few questions I don't quite understand.
Is that "camera description" mandatory ? In other words, given a program who doesn't know the camera beforehand and therefore has to rely on the included data, can it render the image differently from some other programs, depending on its own implementation, or not ?
Or, is the "camera description" so important that will limit the possibilities of different renderings ?
If the program does know the camera, and applies a camera specific profile to the DNG, can it be the same profile applied to the original NEF, or has to be a new one, adapted to DNG, and this situation be the origin of the differences I've found ?
You have to remember that the image data in both the original raw file and the DNG file are the same. The image data is not modified. The only difference is where the metadata is stored.
As I understand it, if ACR recognizes that camera description as one of the cameras that it supports, then it will utilize your appropriate profile and any adjustments you have made to that profile. If it doesn't recognize the camera description it will utilize generic ACR defaults and allow you to adjust accordingly.
In either case, I don't think that limits the possibilities of different renderings. You still have the full array of ACR controls to adjust the image to your liking. Then you can save that adjusted image, complete with specific settings as a separate DNG file.
How well the DNG matches the original raw file I think depends on how completely the DNG specification is implemented in the software.
This probably does not answer your question completely. And I might be in a position to be corrected in what I have written. Hopefully there will be someone who is more knowledgeable than can shed more light on your questions.
Thanks again for taking the time to answer, Jim.
The problem is... ACR, or Lightroom are Adobe's. I fully expect NEF and DNGs from a Nikon camera to render the same in an Adobe product, being DNG an Adobe idea.
In fact, that's what do happen. Both resulting images are equal.
But my doubts come from a recent test of another program, from another company. Phase One's Capture One 4 LE.
There were very noticiable differences on the resulting image. And there comes the doubt.
One can imagine that the generic profile rendering would of course be different than the "direct D200 NEF" and it's specific profile rendering. That's to be expected.
But, what I didn't expect was that the NEF and the DNG would render differently even when using a specific D200 profile. If the RAW data is exactly the same, why the differences ?
That's the reason I'd like to know if it is caused by Phase One's implementation of the Capture One DNG conversion, or if, as the camera description inside the DNG is fixed, it limits in some way the spectrum of possible variations in rendering, and therefore the differences I've observed between a "proper NEF" and a "DNG-from-nef"
The image data is the DNG is identical (bit for bit) to the image data in the NEF, so this sounds very much like a bug in Phase One's software.
It is kind of pointless complaining about Phase One bug on the Adobe users to user forums.
All I can say is that I have created DNG copies of raw images. Then I have made changes to the original raw file, and then copied those changes to the DNG copy and the results have been identical. If you are having problems with the Phase One DNG file, then that would indicate that there is a problem with that software. It goes back to how committed they are to supporting the full DNG specification. There is nothing Adobe can do about that.
This is just my own opinion. But I think we could all spend countless hours comparing different raw converters looking for the one that is going to provide the "perfect" result. Some people seem to be dedicated to proving that there is something better than ACR. And some of them have proven that in their own mind. I think the better approach is to choose a raw converter and then stick with it and learn how to use it. It isn't just a matter of doing a simple conversion like some people think it is. ACR has the capability of producing excellent results. I find that I do 95% of my postprocessing in ACR now. While some may think my approach is rather naïve, it has proven to be very productive for me. I only briefly considered another raw converter, and quickly realized its limitations when compared to ACR. I haven't bothered looking at anything else.
Mr. Knoll -
I'm not complaining about Capture One, here. I talk about Capture One because is the other raw converter I have some experience with, and where I found this situation. Although, for a DNG forum, being DNG a project aspiring to standarization... it shouldn't be a problem.
But the point is not that. I'm just trying to find what's going on with DNG, learn all I can about it, pros and cons, before comitting 80.000 images, and future images, to this format.
Image data, I take for granted is identical. What interests me is if the color space, camera description... I don't know how to call it...the interpretation made by DNG converter or Lightroom of each camera's characteristics in order to allow its fidedign reproduction ... that goes with the raw data in each DNG, may influence in some way the conversion other software may be capable of doing from the image, be it with a generic profile for DNG's or applying specific camera profiles.
On a side note. This info about the camera... it depends on the company behind the software used to convert to DNG, I understand.
If this is right... then in the future, can each software company include it's own interpretation of the same camera? Can it be replaced by a different one, after conversion ?
I don't think there's a perfect image, only renditions you may like more or less.
But that's not the point. I'm not judging ACR nor Lightroom, as I also use them and I'm mostly satisfied. I'm evaluating the limitations this DNG format might impose over my images, if any.
I want it to succeed, but to support it I must know it's pretty and ugly faces before. And that's why I ask :)
Thanks both for your time, and your point of view.
>I'm not complaining about Capture One, here.
Yes you are...
You are asking a question about why C1 is giving different results with an NEF and DNG of the same origin. We don't know...ask Phase One.
The raw data in the NEF is moved over to the DNG bit for bit as Thomas says...therefore, there must be something in C1 that is treating them different because the data is the same. OK?
Carrying out following test *may* help explaining the reason for the different rendering.
Pls download following D200 shot and two DNG files, converted from the same NEF, but the second one a bit manipulated. Check how C1 processes these, then we will see.
Let's clarify something.
It's Adobe, not me, who's claiming and promoting DNG to be a Standard RAW Format.
It's Adobe, not me, who can explain how it works.
Being Standard means also that it must allow other software, not only Adobe's, to render the files as they choose to.
If its Phase One's fault, so be it. But this is not an ACR or Lightroom forum, it's a forum about DNG. I don't need to be kind to Lightroom or Capture One, because in the future if I'm to work with DNG I might choose another different software.
I simply want to know if DNG "per se" will condition the conversion, no more, no less.
So, if there's something in the DNG that might force the renders to be conditioned in certain circumstances, by the description of the camera Adobe DNG Converter (or any other DNG converter) puts into the DNG, I need to know it.
It is not a bad thing nor a good thing. But for an open format...this must be known.
It can be something as simple like I don't understand how DNG works or what's inside it, don't make a war of it.
G Sch -
I've downloaded the files. When I get a minute I'll check it.
Adobe has published the complete specification for the DNG format. They have provided a SDK, free of charge, that anyone can download that will enable them to develop software, plug-in or standalone, to work with DNG files. Adobe does not police these other developers and force them to support the DNG format the same way that Adobe does. As far as Adobe is concerned, the DNG format is a standard or "open" format. But if other software developers choose not to fully support DNG in the same manner that Adobe does, there is nothing that can be done about that as far as Adobe is concerned. If you are having difficulties with another program that was not created by Adobe, then the questions you are asking here should be directed to the other company. As I have stated previously, I can make changes to a raw image downloaded directly from the camera and then save the image with all the changes as a DNG file, and the two images are virtually the same. If Capture One does not produce that result for you then it seems that you need to ask them why that is so.
Well, the world is full of bad implementations of DNG today.
SILKYPIX doesn't render the files the same. They doesn't even look the same in Black and White.
The only progrma so far that renders them equal are Adobe's programs.
What does this means, then ?
By the way, my own files, or the G Sch files that I downloaded... it doens't matter, they look different in DNG and NEF.
Reds are orange-ish, Orange is yellow-ish, blues are darker... in both Capture One and SYLKIPIX.
If I must rephrase it to make it softer, then...I just need to understand why, if the RAW data is the same, and both programs know the D200 camera and NEF files, the problem is not DNG's fault, but the two converters fault.
If there's any other program I should test, please tell me.
Again... I'm asking these questions to them, too.
But... my question is about DNG, not Adobe's programs. It's not my fault if Adobe is DNG's developer.
I may be dumb, and maybe I don't understand how DNG supposedly works. But telling me it's not Adobe's fault won't make me understand it.
If a program bases it's rendering on how it understand a D200 nef has to be rendered (profiles, etc) then why, when the D200 file is in DNG format can't do the same ? Does the program suddenly forgets what the D200 profile is ?
Can the camera description included by the converter on the DNG files cause this ? If not, why ?
>Being Standard means also that it must allow other software, not only Adobe's, to render the files as they choose to.
That's where your expectations are wrong...I don't think there's any guarantee anywhere I've seen that "renderings" of DNG files are expected to be "standard"...can you point one out?
If you understood what a DNG was supposed to do; provide a standardized raw file format (not a standardized raw rendering) then you might understand that EVERY raw processor (except for Camera Raw 4.x and Lightroom) will have a different "default" that is applied to the raw data. But the default (and the built in preview) are essentially meaningless used as a guide from which to adjust an image.
Capture One is not capable of matching Camera Raw's default and visa versa...the renderings of the standardized raw data will vary from processor to processor. The key is that the raw data is all there in the DNG and the format itself is documented and standardized (and open for anybody to use).
An NEF is not...it's neither documented nor open.
I'm starting to think my poor english doesn't allow to convey the info well enough.
I don't expect ACR and SILKYPIX to output the same image.
But I do expect that SILKYPIX and SILKIPIX give me the exact same rendering from the exact same RAW data (bit for bit, quoting Mr. Knoll) ... so, if the raw data is the same, why SILKYPIX can't make a D200 DNG look the same like a D200 NEF?
And the same can be said about Capture One.
It would be a different thing if those programs couldn't render D200 NEFs ... so they'd have to rely on Adobe's D200 calibration.
But they DO know how to render the NEFs. Why don't they know how to render the DNG, if the raw data is exactly the same ?
Are both companies equally unable to match their own D200 profiles ?
Or can't they simply do better because DNG can't be rendered like NEF ?
That's the question.
The easiest answer is that Adobe has shown that a product can render both original camera raw files and DNGs derived from them identically - why other developers don't do this is a question for them; it clearly can be done.
Your question is one that cannot be answered by anyone here. Adobe created the DNG format; using Adobe products, it is possible to get identical results from both original RAW images and DNG images; Adobe has published the file specification and it is available to anyone who wants it. Adobe takes a lot of care when creating new camera profiles that will provide consistent results. If another product does not produce the same results, then you have to ask that company why that is. The image data is exactly the same data. But how the software initially renders that data is under the control of that software program. Adobe has no control over this, and wouldn't want to impose that kind of control.
That's what I first thought, what one can conclude from reading the different articles and posts about DNG. But then I don't understand why even the white balance is different, and I mean the Kelvin degrees that each program show for a NEF and it's derived DNG, because visually both images look right.
So, to clarify ... what all of you are saying is that the camera profile included inside the DNG doesn't affects in any way other software's rendition of the colors, etc. except when that software has to rely on Adobe's calibration because that camera is not supported specifically by the program. Am I right ?
So the next question is ... now the only widely used DNG converter is Adobe's. If DNG is to be a standard someday, there'll surely appear new raw converters (before Each and Every camera maker outputs DNG direct from camera), right ?
Can they include their own camera calibration ?
And, if the answer is yes... then... there'll be so many variants of DNGs (each company's own camera profile included) as DNG converters exists.
How can this affect to standarisation ? Has Adobe any plans on it ?
If I can profile my own camera, and tweak the profile to my liking (supposing I have the tools and the knowledge) ... can I replace the previously included profile ?
Until the digital photography industry decides to standardize, your observations are quite correct. Adobe has attempted to create a standard format, but it's up to other companies as to how they decide to implement that format.
As far as the profile for your camera is concerned, after you have made adjustments and have a standard setting that you like, just save that setting as the new Camera Raw defaults. Those defaults will only be for that specific camera.
Thanks again, Jim.
I'm afraid I'm being a pain in the ***... but my problem has never been ACR, or Lightroom. Since we've purchased a D300, we're working on Lightroom, and I'm mostly pleased with it. I tweak colors when I don't like the default rendering... though I miss more control over the camera profile.
That's why I asked about replacing Adobe's DNG camera profile. If I could tweak the profile,if Lightroom or some other specialised tool allowed me that, and I could include that profile in my DNGs instead of Adobe's , then most of my problems would be gone...well ... if the other programs would then allow me to use it, instead of their own.
That's different from saving Lightroom settings (another benefit of DNG, by the way).
Yes, I'm asking for a miracle.
There is NOT a separate DNG profile for your camera. There is one ACR profile for your camera. The DNG file is raw file. The image data that is in your NEF file and the image data in your DNG file are precisely the same image data, right down to the pixel. The difference in the DNG is where the changes are stored. In the DNG file, the changes are stored in a specific section of the DNG file whereas with your NEF file the changes are stored in a separate "sidecar" XMP file. ACR knows, based on the extension of the file, where to look for the metadata that implements the changes that have been made.
The problem in your case seems to be that the other software does not read and interpret the metadata the same way that Adobe interprets the metadata. Unfortunately, that is something over which you have absolutely no control, other than to complain to the company who isn't doing the job right.
If you create a preset in ACR or Lightroom that produces the "standard" that you are looking for then that preset can be applied to either the original NEF or to the DNG and in Photoshop or Lightroom that preset will produce identical results. That is because it is the same preset, modifying identical data, using the same camera profile. I'm sorry, but I don't know how to state this any more clearly. Somehow you have misunderstood what is happening. I strongly suggest that you purchase a copy of Real World Camera Raw with Photoshop CS3. This is the book that was originally written by Bruce Fraser. Bruce has passed away and Jeff Schewe, who has answered your questions a couple of times in this thread, is the one who revised the book for Photoshop CS3. Believe me, it is a book that is well worth purchasing and keeping close by as a reference.
Very true however it is up to the other companies to do it write. Unless
Adobe is designing other companies software Adobe has no control over how
the decide to handle DNG. In fact most of the other companies only support
DNG files for cameras who's proprietary RAW files they already support. This
is not a requirement, DNG files do not need that to work they software
companies chose to do it that way. That is why when for example the new
Pentax K20D arrives and I shoot DNG files I will be able to open them in
Adobe products. But, I will have to wait to open the DNG files in DXO's,
Capture One, etc's products.
You need to deal with the maker of the software you are having problems
with. Your problem lies with them, not Adobe.
What that means is each company chooses to do things differently. They hope
that it will set the product apart from the others. You see this all of the
time. Some people like the way the camera maker's software renders raw
files. Others like the way Silky makes them, some Adobe, etc. No two
companies are going to render RAW files the same and I know for a fact that
is also applies to non-DNG files. You see the problem besides companies not
implementing DNG correctly is that when it comes to proprietary formats like
NEF each company has to backwards engineer the format and try to figure out
how it works. The makers of NEF Nikon don't spill either any or all of the
beans when it comes to their format. They do this in the hopes they will
have a good reason to sell you there more expensive RAW image processing
software. The problem with this backwards engineering is that each company
is going to get to the support for NEF files in a different way and that
means they will also look different. A similar thing is happening with DNG.
They aren't doing something 100% correct or they are doing something or not
doing something. Again, Adobe has no control over other companies and how
they choose to do things.
You will have to ask the makers of Silkypix and Capture One. The data is the
same for some reason these two companies are handling the data differently.
Sometimes I get the feeling that camera and software companies do everything
they can to try and keep DNG from being as easy as it is. This is why some
programs like DXO's doesn't even support DNG for input. Adobe has no control
over how these other companies handle DNG, this is the part that you need to
Ok, it's not getting through this way. Let me try again.
Adobe Converter includes a camera description, profile, whatever the name, when I convert files to DNG. It is the same profile ACR uses for that camera.
From what I understand, this is done as a way to allow present or future raw developer software to know how to handle that raw data, if they don't have a specific profile for the camera (ie, obsolete, not supported, etc).
If a given software uses it's own "description" of the camera, the included one is not used then.
This is what should be. But not what's happening. It's not Adobe's fault. Adobe is innocent. Free. Not guilty. Pure. Immaculate. Can I stress it even more to assure you I do understand it ?
What I ask is : In the future, when every software developer in the world and the known Universe fully understand how to use DNG properly, am I still bounded by Adobe's interpretation of my camera's response to colors ?
In other words. Does the DNG specification, or Adobe's plans on it, allow me to replace the Adobe included camera "description" by my own (if I ever have the tools - or other software's vendors DNG converter "description").
And a second question. Is that camera description (or the method used to describe it) flexible enough to not limit the spectrum of possible "color customistations" afterwards ?
And, yes, It Is NOT Adobe's Fault.
But it is an Adobe format. So I must ask it here.
Why did not you post the result of the comparison I told you to do?
Ops, sorry. I did check again with your images, and the results were again different, so I thought it was not that important to insist on the fact that the renderings are different.
I finally suppose it has nothing to do with DNG itself, as everyone seems inclined to rememeber me, but with those programs "wrong" way of converting the DNG images.
But, If you're still interested, I'll re-download the images and will post the results -- don't know a good place to post them, but I'll find one.
I don't need to see the result. Only the fact is interesting, if C1 can process the modified version and if yes, whether the result is the same as with the unmodified one.
There is no question about DNG being the "guilty" component or not. It is the business of C1; the question is only the explanation, why C1 behaves differently with the native raw and with DNG.
That was the only thing I wanted to understand.
I've been reading about it, and the answers I've got do point in that direction. Even phase one recognize they are "young" when it comes to DNG support.
Most answers I've got here are just "is not Adobe fault", "it's not DNG fault" or "If ACR does it right, then everything's fine".
The question was wether the color matrix inside the DNG was involved or not. If DNG is to be an standard, then it must work for every developer, and not better with Adobe by default.
So, it seems color matrix can be ignored. But.
I don't like Adobe's color. But I want DNG to succeed.
If at some point in the future I use a program that doesn't know my cameras, and rely on DNG to interpret my files... it will use Adobe's interpretation, wich I suppose I will be able to tweak in some way if that imaginary program has the tools for it.
And that's the new question.
Will Adobe's calibration of my camera limit in some way the spectrum of tweaks I might be able to apply to the files in that hypothetic situation ?
Can the "profile" be tweaked, or substitued, now or in the future ? I don't mean afterwards, I mean the one that gets embedded in the camera when converting to DNG. And NO, ACR settings are not what I'm talking about.
Thanks all for your patience and your interest...and your time.
What was the result of the test I asked for? You take a considerable time for answering a VERY SIMPLE question.
As SIMPLE as READING my answer, I guess. Answer 10.27am Feb 10.
The results were different. And that implies C1 could process both files.
Honestly, I can infer from your posts by SIMPLE READING only that you are confused and speculating a lot.
I try to put the results in cleartext:
1. C1 creates result A from the native raw file.
2. C1 creates result B from the DNG file converted from the raw. B is different from A.
3. C1 created result C from the modified DNG file. C is different from A AND from B.
If the above is right, then we can say, that
a. C1 CAN work with the pure DNG data, which includes the color conversion Adobe's way. The modified DNG differs from the original DNG in that the camera name (maker and model) has been changed, AND the MakerNote tag has been made unaccessible, and this file has been accepted by C1.
This is in contrast to some other raw processors, which accept a DNG file only if they know the camera's characteristics; see the thread "Is DNG a universal raw format?" down below.
b. C1 DOES take some information from MakerNote if available, otherwise these two version would yield the same result.
The originating question was, why the "native" and DNG result are different. Based on the above test, I can not answer it with certainty, it is not clear, what C1 uses from MakerNote.
Now, to the question of "DNG - standard or not".
The DNG specification tries to put many aspects of a raw image in a uniform shape. However, there are two distinct areas posing problems:
1. Camera specific options. The Exif specification provides for a way of recording sharpness, contrast, saturation and white balance, but the specification has been created very short-sightedly and became useless. Other options, like Picture Style has not been envisioned at all. Such options are coming in series, the DNG specification does not cover them.
These options are available in the native raw file as well as in DNG, but programs have to make extra gymnastics to extract them from DNG; NOT DNG is to blame in this point.
Some raw processor go the extra way, others don't. ACR does not make anything, which can not be seen in DNG format as well, i.e. it does not support these options.
2. The other problem area is the color transformation. There is *no* generally accepted way to describe the sensors' behaviour. Most raw processors adopted camera and model specific solutions; Adobe defined a way in DNG, which is an *approximation*. Others don't share the enthusiasm, with some justification. It is not by chance, that camera makers' own converters give the best colors of their cameras.
So, a raw processor can process a DNG file using the hard-coded knowledge of the sensor - like Aperture does - or going ACR's way of color transformation.
C1 may take some setting(s) from the native raw file, while not doing so with a DNG, even if the data is available (but differently). This may be the reason for the difference between the results of the NEF and DNG conversion.
Note, that it is not necessary to access the MakerNote for identifying the camera.
Very interesting info, G Sch. I'm starting to see the light ;)
I thought it was clear from the start I am confused and speculating ;)
Yes, I don't know yet what to make of all this different, often incomplete, and sometimes opposing information I can find on this matters... in "layman terms", I mean. Technical notes may well solve most of my questions, but I'm not still able to decrypt them :)
About camera specific settings, or maker notes in general... I suppose it will come the time when software developers will be able to handle DNG properly and extract the info inside. Extra gimnastics, as you well put it, but nothing else.
The problem that concerns me the most is color. I prefer other software's renderings of my camera colors. So I'm just supposing that, in the future, when/if my cameras go unsupported, the color I'll get from DNG will be conditioned by Adobe's (DNG) method of "describing" the sensor.
Even if specific tools to tweak the color are available I don't know if the spectrum of tweaks, the possibilities to alter the color range, will still be conditioned by the method used to describe the sensor. (Sorry, don't know how to express it better in English)
That's one of my questions (speculative, again) ... will it be possible to alter the method used by adobe ?.
Has the method flexibility enough to be adapted to my particular taste, or even camera makers requirements, or will it need to be substitued alltogether by a new and different one ?
And, if all of this happens, or not, would be possible for users to customize the "sensor description" included in DNG in some way to adapt the default rendering to our own "taste" ?
I ask this because the way things look to me, if camera makers or users aren't happy with the "look" they get form their images or the ability of this method to describe the "sensor character" closer to what the camera maker/owner wants, DNG won't be a standartd. It needs to keep "everyone happy", and doing it in a standard format. It is not a solution to tell the rest of the world "this is adobe's way, adapt and sacrifice or leave".
>So I'm just supposing that, in the future, when/if my cameras go unsupported, the color I'll get from DNG will be conditioned by Adobe's (DNG) method of "describing" the sensor
One reason of keeping competition high is, that even if Canon stopped supporting some old cameras for example in Windows 2020, some competitor would jump in the niche. (Of course that too would not be equal to the manufacturer's own product, but it could come closer to that.)
>would be possible for users to customize the "sensor description" included in DNG in some way to adapt the default rendering to our own "taste" ?
Actually, in terms of customization ACR is the best. Unfortunately, the best is not always good enough, as the limits of this customization are set by the underlying method.
On the other hand, the MakerNote does not contain a better characterization of the sensor either. In fact only some cameras' MakerNote contains some sparse information. The raw processors, even the camera manufacturers' own, have to rely on the specific knowledge of a particular camera.
IMO the camera manufacturer independent raw processors should provide some interface (like codecs or otherwise) to accomodate new cameras and third (or fourth) party software. For example Raw Shooter, my favourite raw developer of the past, sucked in colors, but it offered an interface for color profiles, which could be purchased separatedly.
That interface for color profiles... don't know how it was with Raw Shooter, but if it worked like the one in Capture One Pro, it's one of the (almost ignored) feature requests I've made for Lightroom. With this tool I can tweak at will the default response I'll get from my camera when editing the raws. It would be very very useful, in my opinion, if Lightroom could include such thing.
On the other hand, when I talk about customising DNG ColorMatrix, what I'd want to do is modify it, and re-save into DNG. Or create a custom one and include it by default instead of Adobe's one. A pre-customization of sorts vs. tweaking the image once opened.
Though, it seems Lightroom ignores some of the info inside DNG and relys on Camera Model instead... it should be different for DNGs, or that's what I expected.