I'd suggest that easist route to what you want is to first convert your images to linear raw DNGs, then manipulate the linear raw data. That data is demosiaced, but still linear, without tone curves, etc. Also, the DNG format is well documented, and you can use the DNG SDK access the data, rather than having to write your own code.
I agree with Sandy. This is the best way to go.
Thanks guys; that sounds exactly what I'm after. I'll have a play with generating linear raw DNGs tonight. Can I assume that producing such a DNG from a RAW, and loading it back into ACR, would give me identical results to adjusting the sliders on the original RAW file (i.e. when loading a RAW, does ACR produce the same linear raw data and adjust that using the sliders)?
only if linear DNG is produced using the same version of ACR... Eric Chan commented before that Adobe DNG converter does not use the same demosaick as ACR/LR (that was during the discussion that you can actually use Adobe DNG converter as a free command line raw converter (to JPG - extract JPG preview from DNG afterwards), if you find a way /type it manually if you want/ to produce raw conversion parameters in xmp for example)...
also this posting and below = http://www.luminous-landscape.com/forum/index.php?topic=38898.msg560488#msg560488
Interesting, thanks. I don't have Photoshop in front of me right now, but could you use the ACR dialog in Photoshop CS6 to save a linear raw DNG (to ensure it'd be exactly the same as using ACR with a RAW)?
It does seem odd to use a different demosaic algorthim though. Is it a case that they might be different, but if you have an up-to-date ACR plugin for Photoshop, and the most recent DNG converter, they'd be the same?
> Clearly it would be time-consuming to reverse engineer a camera’s RAW format myself
not at all - the work is done (dcraw) and even refactored in a more human readable form (C++ library) ready for consumption (libraw.org)... plus there are compiled versions (of dcraw or some examples in libraw) that can produce you what you need... if you run mac (or osx on pc) you can use converter like RPP that can generate linear output, even in camera's RGB (not in color space if you wish so), 16 or 32 bit.
> but could you use the ACR dialog in Photoshop CS6 to save a linear raw DNG
yes - click "Save Image..." button (bottom left) and set custom options for DNG
> It does seem odd to use a different demosaic algorthim though. Is it a case that they might be different, but if you have an up-to-date ACR plugin for Photoshop, and the most recent DNG converter, they'd be the same?
that was exactly what Eric Chan said - demosaick in DNG converter (not to mix that with a source code of DNG SDK) is not the same... Adobe DNG converter replicates everything else, including local edits when generating an image preview to embed in DNG file, but demosaick was said not to be the same... but the resulting output is JPG image anyways, so you might not be able to find a difference (visually).
PS: ____unless____ it is the same demosaick to create a linear representation, but not the same demosaick to create a JPG preview... sounds like a perversion, but you never know
I believe that's a miscommunication. The DNG Converter does use the same demosaic processing as ACR/LR (assuming matching version). It's the public DNG SDK that uses a different (simpler) demosaic processing.
Yes, producing a linear DNG, then re-loading that DNG into ACR and processing it, will produce the same results, assuming all the relevant metadata is intact.
There is more to decoding a raw format than simply reading the pixel data. (dcraw gets the image data and some metadata like the as-shot WB, but there is also a lot of important private metadata that it does not read.)
> I believe that's a miscommunication.
Thank you for clarification and correction ! My memory failed me ...
That also means that DNG Converter is 100% capable (assuming the same version) to reproduce parametric adjustments stored in xmp (or dng) by ACR/LR when creating a jpg preview to store in DNG file... I was for whatever reason that you noted before that it is not the case and the reason was the demosaick - that is why I was under such wrong impression about differences between DNG Converter and ACR/LR.
Many thanks all for the info.
I didn't want to use a "3rd party" demosiac system (such as dcraw) as I was trying to get the same results as ACR. Knowing now that I can write linear DNG files from ACR or the DNG Converter and get the same results as the original RAW is great. I haven't tried the DNG Converter recently, but I did experiment with creating a few linear DNG files from ACR (v7.1) in CS6 last night, and can confirm that, as expected, they give identical behaviour to loading the original RAW (even with extreme slider adjustments).
I presume there's enough command line support in the DNG Converter to allow me to automate/script the generation of a bunch of linear DNG files from RAWs, then access the not-quite-RGB (http://www.openphotographyforums.com/forums/showpost.php?s=91e4600b76e07efaf2acb294de7f559 f&p=104537&postcount=1) data using the DNG SDK to write out a single blended linear DNG. The fact the public SDK has a simpler demosaic algorithm is of course not a problem in this instance, as I'd only be dealing with already demosaiced data.