37 Replies Latest reply on Jan 30, 2017 5:51 AM by Bjornago

    Color flickering in sequence of identicially processed raws

    gmyny Level 1

      I have a time-lapse sequence taken with my Olympus E-M10, shooting ORF files. About 10% of the pictures show a different white balance and tint, resulting in a noticeable flickering. The whole sequence was processed in Lightroom with the exact same values.

       

      I looked into the EXIF data and it seems that all values relating to colour balance are identical for all pictures.

       

      Now there are two possibilities (as far as I can see):

      1. Lightroom does, from time to time, something weird while importing the pictures.

      2. The data in the raw files is already different. I have no idea what could cause such a behaviour in the sensor or in-camera processor.

       

      Any hints, experiences or advice on how to investigate this problem further are highly appreciated.

       

      Cheers

      gmyny

        • 1. Re: Color flickering in sequence of identicially processed raws
          ipsilanti Level 1

          Having the same problem with Olympus cameras (E-PL5, E-M1, E-M5 II) and thought this is a problem of Olympus. It is the serious problem, because I'm shooting timelapces and such colored flicker is very difficult to smooth. And I found that it occurs when I'm making more than +2EV stops compensation.

           

          Yesterday I took 2 sample RAW-files which has identic exposure but looks different in Lightroom and opened them in Olympus Viewer 3 (official software from Olympus). So they was looking absolutely identic. And this is because now I'm thinking this is a problem of Adobe.

          • 2. Re: Color flickering in sequence of identicially processed raws
            johnrellis Most Valuable Participant

            Why don't you upload the two samples to Dropbox (or similar) and post the sharing links here.  Then some of the experts here can take a look and determine whether this is a bug in LR, and either help you post an authoritative bug report and/or figure out a way to avoid the issue.

            • 3. Re: Color flickering in sequence of identicially processed raws
              ipsilanti Level 1

              johnrellis frankly speaking I thought it would be no profit of that. The topicstarter gmyny made his post 2 mounth ago and there was no any reaction. But I can try..

               

              Here are 2 RAW files with identic exposure but different colors. I left XMPs and also made lowres JPGs.

              https://dl.dropboxusercontent.com/u/48960608/color_flicker.zip

              • 4. Re: Color flickering in sequence of identicially processed raws
                johnrellis Most Valuable Participant

                It appears from the JPEGs you are using LR 6.4 on Mac?

                 

                I imported the two .orf raw files with the .xmp sidecars and then imported copies without sidecars, so I had four .orfs.  I noticed that the two developed raw files had different Tint values (21 and 31).  That would certainly account for different white balance, but figured it might have been your attempt to smooth out the color difference.

                 

                So I tried copying all the develop settings (including Camera Calibration, which has non-default settings for the color sliders) from the developed M5266038.ORF to the two undeveloped raws.   And doing that, I can clearly see that the colors are much different even though the develop settings are exactly the same, as you describe.

                 

                Next, I reset the undeveloped raws and just changed their exposure to 2.65, leaving the process version at 2012. I can see a noticeable difference in the color.

                 

                I saw two things about the undeveloped raws: One has Tint = 21, and the other Tint = 22.  And their histograms are noticeably different on the left end:

                So even before applying any develop settings, the pics have different colors, even though the difference between the two appears to be an insignificant shift of the clouds.

                 

                I don't know if this difference is a result of what the camera recorded or of LR's initial processing or both. This is beyond my expertise, but maybe trshaner can enlighten us.

                • 5. Re: Color flickering in sequence of identicially processed raws
                  gmyny Level 1

                  I also opened a ticket here:

                  Camera Raw/Lightroom: ORF white balance inconsistent | Photoshop Family Customer Community

                   

                  I included sample pictures and there was some discussion, but also no proper conclusion. ipsilanti please feel free to dig into this more. Maybe it helps if more than one person sees the exact same behavior.

                  I also was in contact with Olympus support. They obviously argue that all looks good with the Olympus Viewer, so it's not their problem... .

                  Cheers

                  gmyny

                  • 6. Re: Color flickering in sequence of identicially processed raws
                    ipsilanti Level 1

                    Here is the editing result in Olympus Viewer 3 with the same adjustments (+2EV, 3100K, Tint +2):

                    https://dl.dropboxusercontent.com/u/48960608/Olympus%20Viewer%203.zip

                     

                    As I told they look absolutely the same.

                     

                    Also I tried to open RAWs in "DxO OpticsPro 11" and found the same color flicker as in Adobe. But I think this app uses the same algorithm, because its close integration with Lightroom.

                     

                    Maybe I should try any editor based on dcraw?

                    • 7. Re: Color flickering in sequence of identicially processed raws
                      ipsilanti Level 1

                      Well pals, the problem looks much more serious than I thought.. I made my investigation and here what I found.

                       

                      1. I decided to try my RAWs with any editor based on DCRAW as open source project. I chose RawTherapee for Mac and found the same problem with color flicker. I will not share JPGs here, they are almost the same as in Lightroom.

                      2. And what do you think? Then I found that Adobe using the same open source DCRAW codes! dcraw?

                       

                      So, now I realize that problem is much bigger and there are 2 possible reasons:

                      1. Olympus indeed making different RAW files and somehow Olympus Viewer 3 understands these differences and making final images the same (actually I can't imaging how, but..).

                      2. The initial problem is in DCRAW. And all software that use these source codes are making images with color flicker. If that is true, I understand that technically it is not Adobe fault. But in the other hand I'm using Adobe Lightroom as a final product and it should work correct.

                       

                      Does anyone can advise any software that uses different than DCRAW source codes?

                      • 8. Re: Color flickering in sequence of identicially processed raws
                        Bob Somrak Level 5

                        ipsilanti wrote:

                         

                        Then I found that Adobe using the same open source DCRAW codes! dcraw?

                         

                         

                         

                        That thread is from 2008!!!!!

                        • 9. Re: Color flickering in sequence of identicially processed raws
                          johnrellis Most Valuable Participant

                          Olympus indeed making different RAW files and somehow Olympus Viewer 3 understands these differences and making final images the same (actually I can't imaging how, but..).

                          I think it's pretty clear from your samples and gmyny's samples that the two raws are different and that difference is causing LR to generate visually different colors whose difference is magnified when you increase exposure.  The questions raised by both of you is why does LR do that, while the Olympus Viewer 3 does not, and should LR be doing that?

                           

                          I suggest you post your results in gmyny's thread in the official Adobe feedback forum: Camera Raw/Lightroom: ORF white balance inconsistent | Photoshop Family Customer Community . That is where Adobe wants bugs, possible bugs, and feature requests to be posted.   Adobe product developers actively participate in that forum, but they are rarely seen in this forum. 

                          • 10. Re: Color flickering in sequence of identicially processed raws
                            trshaner Adobe Community Professional & MVP

                            johnrellis wrote:

                             

                            So even before applying any develop settings, the pics have different colors, even though the difference between the two appears to be an insignificant shift of the clouds.

                             

                            I don't know if this difference is a result of what the camera recorded or of LR's initial processing or both. This is beyond my expertise, but maybe trshaner can enlighten us.

                            The two .orf files have identical exposure, which is ~2.5 EV underexposed. Because of this LR should process the two files identically concerning highlight recovery etc. What I did notice is that Auto White Balance was used, which is probably not a good idea for this type of time lapse photography. Best suggestion is to use a fixed in-camera white balance setting such as Daylight or 5,200K.

                             

                            Exposure Program           : Manual
                            White Balance              : Auto
                            White Balance 2            : Auto
                            White Balance Temperature  : Auto
                            • 11. Re: Color flickering in sequence of identicially processed raws
                              johnrellis Most Valuable Participant

                              Basic question: How does the in-camera white balance affect LR raw processing?  Isn't the raw pixel data the same regardless of which in-camera white balance is selected?

                              • 12. Re: Color flickering in sequence of identicially processed raws
                                ipsilanti Level 1

                                That thread is from 2008!!!!!

                                Bob Somrak So?? Does Adobe changed source codes?

                                 

                                Best suggestion is to use a fixed in-camera white balance setting such as Daylight or 5,200K.

                                trshaner I think there is no difference when we talking about RAW. Isn't it?

                                 

                                johnrellis Definitely! I think the same.

                                • 13. Re: Color flickering in sequence of identicially processed raws
                                  trshaner Adobe Community Professional & MVP

                                  johnrellis wrote:

                                  Isn't the raw pixel data the same regardless of which in-camera white balance is selected?

                                  Yes that is correct. Looking at the OP's two raw image files we see differences due to lighting changes. The exposure is set to manual so that isn't changing, but the Auto 'As Shot' image white balance is slightly different. No Auto White Balance Algorithm is perfect, but it gets you in the ball-park. It works well when the lighting color temperature is "uniform" across the entire image (i.e. all direct sun or tungsten light). In the OP's two .ORF raw files we have a normal color temperature subject (building and clouds) and a much higher color subject (clear blue sky). Auto White Balance isn't going to work well with this type of subject composition. ERGO- My suggestion to use a fixed in-camera white balance setting like Daylight or 5,200K.

                                   

                                  However, it's obvious something more than the WB setting is causing a difference in the rendering of the two slightly different .ORF raw files. I examined the .ORF EXif data and found a difference in the 'Black Level 2' readings as below:

                                   

                                  File Name                  : M5266035.ORF
                                  Black Level 2              : 252 252 252 252
                                  File Name                  : M5266038.ORF
                                  Black Level 2              : 252 253 253 252

                                   

                                  Using RawDigger I changed the Black Level from 252 253 253 252 to All 252, same as the M5266035.ORF file. Guess what? The RawDigger exported TiFFs look identical once they are white balanced inside LR. CORRECTION: So the issue is that LR and RawDigger (DCRAW) do not use the .ORF raw file Black Level 2 setting properly.....but Olympus' Viewer 3 does. RawDigger's Rendering isn't the greatest, but it proves the root cause of this issue!

                                  RawDigger Black Level Adjust To 252 All.jpg

                                  • 14. Re: Color flickering in sequence of identicially processed raws
                                    ssprengel Adobe Community Professional & MVP

                                    If the images look the same in RawDigger when you set the black levels all the same then it would seem that LR is using the varied black levels as specified by the camera but the Olympus Viewer is not so its images look more the same.

                                     

                                    So to me the root cause is that the black levels embedded by the camera are different.  And LR is correct using those different black levels since how would it know any better.  The question is what the Olympus viewer is doing when the black levels are different, yet it's results appear the same.

                                    • 15. Re: Color flickering in sequence of identicially processed raws
                                      johnrellis Most Valuable Participant

                                      If the images look the same in RawDigger when you set the black levels all the same then it would seem that LR is using the varied black levels as specified by the camera but the Olympus Viewer is not so its images look more the same.

                                      Aren't there two plausible scenarios:

                                       

                                      Scenario 1: The luminance distribution of the two raws is different, and the different black levels reflect that.   Olympus Viewer is obeying the camera-recorded black levels and displays the photos the same; LR ignores the black levels and displays them differently.

                                       

                                      Scenario 2: The luminance distribution of the two raws is the same, but for some reason the camera recorded a different black level for the second raw.  Olympus Viewer ignores the camera-recorded black level and displays the raws the same; LR pays attention to the black levels and displays them differently.

                                       

                                      In gmyny's situation, I think you showed that the color histograms were different, so that suggests scenario 1.  For ipsilanti's case, LR's post-profile histograms are different, but I don't know if that necessarily implies the raw histograms are different if LR is applying different internal settings.

                                      • 16. Re: Color flickering in sequence of identicially processed raws
                                        johnrellis Most Valuable Participant
                                        Auto White Balance isn't going to work well with this type of subject composition. ERGO- My suggestion to use a fixed in-camera white balance setting like Daylight or 5,200K.

                                        Alternatively, couldn't you just change the Develop white balance from As Shot to Daylight?  (As you've explained, this doesn't account for the difference's we're seeing in ipsilanti's photos, but I just want to make sure I understand this.)

                                        • 17. Re: Color flickering in sequence of identicially processed raws
                                          ssprengel Adobe Community Professional & MVP

                                          Setting the black levels the same yields similar looks in RawDigger.  That suggests Scenario 2 is what is happening.

                                          • 18. Re: Color flickering in sequence of identicially processed raws
                                            trshaner Adobe Community Professional & MVP

                                            The Olympus micro 4/3 ORF raw file uses 12bit precision image data. The difference between a 252 and 253 Black level value is very small.

                                             

                                            12-bit data values = 0-4095

                                             

                                            1/4096 = 0.0002441 = .02441%

                                             

                                            LR’s Histogram only displays to one decimal place (xx.x%) so this value is too small to be measured inside LR. It should have no visible change to the image inside LR or any other viewer. So what’s happening?

                                             

                                            One possible scenario is that LR (and DCRAW) is assigning the wrong bit depth to the ORF Black Level value. Let’s assume LR incorrectly treats it as a normal 8-bit value and not the actual12-bit value.

                                             

                                            8-bit data values = 0-255

                                             

                                            1/255 = .003922 = .3922% = ~ 0.4% in the LR histogram

                                             

                                            This is probably just enough to cause the “uneven” WB shift in the Ops ORF files. But why is the WB different across the image? Shadow area pixels (deep blue sky) have smaller data values and are offset by a larger amount than brighter area pixels that have larger data values. 

                                             

                                            I tested this with a Canon 5D MKII CR2 file, which uses 14-bit precision image data.

                                             

                                            14-bit data values = 0-16383 (4x the ORF raw file)

                                             

                                            Using a 4x offset value (253-252 = 1 x 4 = 4) I changed the Black Level values in RawDigger from 1024, 1023, 1023, 1023 to 1024, 1027, 1023, 1027. As expected there is no visible difference in the image.

                                            RawDigger Black Level Auto vs +4 G and G2 Offset.jpg

                                            • 19. Re: Color flickering in sequence of identicially processed raws
                                              johnrellis Most Valuable Participant

                                              Setting the black levels the same yields similar looks in RawDigger.  That suggests Scenario 2 is what is happening.

                                              Make sense.

                                              • 20. Re: Color flickering in sequence of identicially processed raws
                                                johnrellis Most Valuable Participant

                                                The difference between a 252 and 253 Black level value is very small.

                                                I wonder if that difference is in fact sufficiently large to cause the effects we're seeing. 

                                                 

                                                I exported screenshots from RawDigger of M5266038 with its original black levels and with levels = 252, 252, 252, 252.  I loaded them into Photoshop as layers and set the blend mode to Difference, which is visible (barely) on my calibrated display

                                                Flattening the layers and applying Levels gives a rough quantification of the difference (green pixel values differing from 1 - 8 out of 255, 8/255 = 0.031):

                                                RawDigger's linear histogram of the green channel is:

                                                (Red and blue look similar.)  Almost all the pixels have green values in [0..80]. 

                                                 

                                                If the white-balance algorithm looks at relative pixel values (min to max), then certainly 1 part out of 80 seems significant (1/80 = 0.0125, 8/255 = 0.031).

                                                 

                                                Big hand wave here, since I lack basic literacy on the topic.

                                                • 21. Re: Color flickering in sequence of identicially processed raws
                                                  trshaner Adobe Community Professional & MVP

                                                  johnrellis wrote:

                                                   

                                                  The difference between a 252 and 253 Black level value is very small.

                                                  I wonder if that difference is in fact sufficiently large to cause the effects we're seeing.

                                                   

                                                  I exported screenshots from RawDigger of M5266038 with its original black levels and with levels = 252, 252, 252, 252. I loaded them into Photoshop as layers and set the blend mode to Difference, which is visible (barely) on my calibrated display

                                                   

                                                  There are some issues with your test.

                                                   

                                                  GEEK WARNING: This may be more information than you want (or need) to know.

                                                   

                                                  1) LR and RawDigger both use DCRAW and if it's not processing the Black Level value properly then you should see a BIG difference. Correct?

                                                   

                                                  2) You're using screenshots, which are 8-bit monitor profile color space. It's going to show the difference, but not very accurately. The Research and Profile editions of RawDigger include Export to TIFF file, which is what I used in my Canon 5D MKII posted example.

                                                   

                                                  3) Using Layer Difference and then expanding it using Levels is simply "amplifying" the difference. A better way to visually check the difference in PS is to open the two files as layers and then use the eyeball icon on the top layer to toggle it ON/OFF.

                                                   

                                                  It comes down to simple math (doesn't it always). You might want to read the article at the link I posted to better understand the metrics of A/D converters and raw file data bit precision: Canon DLC: Article Print: 14-bit A/D Conversion in DIGIC 4 Cameras

                                                   

                                                  Olympus ORF Raw File: 12-bit Precision raw data with 0-4095 image data values (4096 total values)

                                                   

                                                  Black Level Readings of 253-252 = data value 1

                                                   

                                                  1/4096 = 0.0002441 = .02441%

                                                   

                                                  I don't mean to sound trite, but it's really just simple math. Choose any of your own raw files that has obvious highlight clipping and open up two instances of RawDigger with the same image file. Apply that 1 data bit difference to the Black Level and leave the other on Auto. You'll need to scale up that value to 4 if your camera's raw file's are 14-bit precision.

                                                   

                                                  14-bit precision data values = 0-16383 (4x the ORF raw file)

                                                   

                                                  To determine what data precision your camera's raw files use look at the maximum highlight value:

                                                   

                                                  <16383 but >4096 = 14-bit precision image data

                                                  <4096 = 12-bit precision image data

                                                   

                                                  You can also see this in above the image in RawDigger:

                                                  The Canon Max readings show clipped highlights and the maximum A/D output value, which is always less than the maximum range (16383 for 14-bit precision).

                                                   

                                                  Now compare these two images as rendered in RawDigger. There should be no "visual" difference.

                                                   

                                                  • 22. Re: Color flickering in sequence of identicially processed raws
                                                    trshaner Adobe Community Professional & MVP

                                                    johnrellis wrote:

                                                     

                                                    Auto White Balance isn't going to work well with this type of subject composition. ERGO- My suggestion to use a fixed in-camera white balance setting like Daylight or 5,200K.

                                                    Alternatively, couldn't you just change the Develop white balance from As Shot to Daylight? (As you've explained, this doesn't account for the difference's we're seeing in ipsilanti's photos, but I just want to make sure I understand this.)

                                                    Yes that is correct.......if you never shoot in-camera JPEGs. IMHO when shooting JPEGs or raw+JPEG you should manually set the camera's WB for each scene lighting change (Daylight, Flash, Tungsten). The Auto WB setting will do this for you, but you're going to have WB variations "baked into" the JPEG image data due to subject color differences. How well it works depends on the WB algorithm the camera is using, but it will always be influenced (i.e. shifted) to some degree by large color areas (blue sky, red dress, etc.). BTW-The LR Auto WB setting uses a very simple algorithm........it's generally much worse than the in-camera Auto WB.

                                                    • 23. Re: Color flickering in sequence of identicially processed raws
                                                      johnrellis Most Valuable Participant

                                                      Sorry, I didn't explain myself very clearly.  While a difference of 1 in black level is only 1 part out of 4096 possible linear pixel values, the photos in question only have a linear range of about 0 to 80, according to RawDigger:

                                                      RawDigger and LR are only showing that range of values by setting the effective white and black points. So changing the green black level from from 253 to 252 would effectively shift the green distribution to the right in the displayed photo by 1 part out of 80 (0.0125), rather than 1 part out of 4096 (0.00024).  We'd expect this shift to result in slightly brighter greens.

                                                       

                                                      I can see this in the PSD I made of the two versions of M5226038.ORF, one with green black levels of 252 and one with 253 (without the difference blending mode).  When I click the top layer on and off, I clearly see a green shift, especially in the blue sky.  (I should have mentioned this yesterday, before trying to quantify the amount of shift by taking the difference.)

                                                       

                                                      The shift in color is very similar to the shift I see in LR between the two photos when I change their Develop exposure to 2.7 (which effectively sets the black and white levels to roughly [0..80]).  The LR histograms of these photos show the shift in green:

                                                      I estimated the magnitude of the green shift by pasting the histogram into PS and using its ruler.  The left edge of the green bump shifts 3 to 4 pixels, out of a total range of 320 pixels from left edge to right edge of the distribution.  That's a relative shift of 3/320 = 0.009 to 4/320 = 0.0125, which corresponds to what I estimated from RawDigger's histograms.  (LR may not be using a linear transformation on the distributions, but I'm just estimating orders of magnitude for judging plausibility.)

                                                       

                                                      So from the observations you and ssprengel made, it sure seems that the issue is that the camera is recording a slightly different green black level for the second photo, and that LR and RawDigger are correctly interpreting that difference.  When the Develop exposure is increased in LR, the difference in green black level is enough to cause a noticeable green shift.  

                                                      • 24. Re: Color flickering in sequence of identicially processed raws
                                                        johnrellis Most Valuable Participant

                                                        Thanks for the additional details about in-camera and LR white balance.

                                                        • 25. Re: Color flickering in sequence of identicially processed raws
                                                          trshaner Adobe Community Professional & MVP

                                                          johnrellis wrote:

                                                           

                                                          Sorry, I didn't explain myself very clearly. While a difference of 1 in black level is only 1 part out of 4096 possible linear pixel values, the photos in question only have a linear range of about 0 to 80, according to RawDigger:

                                                           

                                                          RawDigger and LR are only showing that range of values by setting the effective white and black points. So changing the green black level from from 253 to 252 would effectively shift the green distribution to the right in the displayed photo by 1 part out of 80 (0.0125), rather than 1 part out of 4096 (0.00024). We'd expect this shift to result in slightly brighter greens.

                                                          I suspect DCRAW is not applying the Black Level values correctly and in effect "over-correcting" the G G2 image data Black Level 0 you see in the RawDigger Histogram. If this is true the G and G2 pixel data just above the 0 level will look different. So what you are seeing is expected (over-correction) and causing what the OP is seeing. Keep in mind there are real WB differences in these two raw files, which will make the histograms look different.

                                                           

                                                          The sensor output has a DC bias voltage that sets the 0 level (Black Level). This DC bias voltage varies with temperature and the R, G, B, G2 outputs may have different values. The Black Level readings are used to normalize the image data values by very slightly changing the 0 level (252, 253) as follows in the OP's ORF files:

                                                           

                                                          Data Values 0-100%

                                                          252-4096

                                                          becomes

                                                          253-4096 for the G and G2 outputs

                                                           

                                                          This corrects for non-uniform DC bias voltage drift. In effect the G and G2 0 level outputs have been detected as drifting slightly to 253. This is monitored using border located sensor sites that are out of the light path (dark rows). The 253 Black Level lowers the G and G2 outputs by subtracting the detected DC offset amount (data value 1), which is still only 1/4096 and very small.

                                                          • 26. Re: Color flickering in sequence of identicially processed raws
                                                            trshaner Adobe Community Professional & MVP

                                                            To actually test your theory in post #23 I used a raw image file from my Canon 300D, which also has 12-bit precision data. It's Black Levels are at 132 131 131 132, or about 1/2 the ORF file. The image requires +3.5 Exposure in LR, which is a little more under-exposed than the OP's ORF files. So this should be a slightly worse-case image. Using RawDigger without 'Automatic Exposure Correction' I Exported an Auto Black Level TIFF and one with G and G2 raised by  (132 132 131 133). The required WB settings were virtually identical when sampling a white area (-19, 30), but slightly different on a black area. It's very slight and expected since the G and G1 levels have been altered and the amount of shift appears smaller than observed in the two ORF files.

                                                            Canon 300d -3.5 EV Underseposure Black Level Test.jpg

                                                             

                                                            That said your theory may in fact still be correct since there are lot's of other variables  that come into play.

                                                             

                                                            SUGGESTION TO

                                                            • 27. Re: Color flickering in sequence of identicially processed raws
                                                              johnrellis Most Valuable Participant

                                                              I confirmed that Olympus Viewer 3 entirely ignores the MakerNotes field Black Level 2, whereas RawDigger and LR don't. Using Exiftool, I changed Black Level 2 to "100 100 100 100", and it had no impact on the displayed image in OV3, whereas it did in RawDigger and LR. I thought perhaps that the black level might also be encoded elsewhere in the raw file in a spot invisible to Exiftool, so I used a binary editor to search for various encodings and orderings of "252 253 253 252" (8 bit, 16 bit, 32 bit, ASCII), but didn't find them.

                                                               

                                                              This explains, as previously hypothesized, why there is no color shift in the photos when viewed in OV3 -- it's not using the recorded black level, unlike LR and RawDigger.

                                                              • 28. Re: Color flickering in sequence of identicially processed raws
                                                                trshaner Adobe Community Professional & MVP

                                                                johnrellis wrote:

                                                                 

                                                                This explains, as previously hypothesized, why there is no color shift in the photos when viewed in OV3 -- it's not using the recorded black level, unlike LR and RawDigger.

                                                                Thanks for confirming. Hard to believe Olympus' own software wouldn't do Black Level correction. It's a basic operation that all raw editors apply during demosaicing. I wonder if Olympus Viewer 3 checks the image data for under-exposure and uses a different algorithm to apply Black Level correction if it falls below a certain threshold? All Olympus micro 4/3 cameras have 12-bit precision raw data, which would be prone to this issue. Cameras with 14-bit raw data are probably not affected with this issue because the Black Level is calculated using two extra bits (4x the precision).

                                                                • 29. Re: Color flickering in sequence of identicially processed raws
                                                                  johnrellis Most Valuable Participant

                                                                  I downloaded a sample raw for the same camera, normally exposed in daylight:

                                                                  Same results -- changing Black Level 2 to "100 100 100 100" or "500 500 500 500" didn't change the displayed image or the exported TIFF.

                                                                   

                                                                  Sure seems that Olympus Viewer 3 is just ignoring the black level recorded in MakerNotes.  Maybe there's a hidden field also recording the value, but I couldn't find it.

                                                                  • 30. Re: Color flickering in sequence of identicially processed raws
                                                                    trshaner Adobe Community Professional & MVP

                                                                    johnrellis wrote:

                                                                     

                                                                    I downloaded a sample raw for the same camera, normally exposed in daylight:

                                                                    Same results -- changing Black Level 2 to "100 100 100 100" or "500 500 500 500" didn't change the displayed image or the exported TIFF.

                                                                    You won't see any color change if the values are all equal. Try adding values that change the R G B G2 color balance, such as below:

                                                                    252 252 252 252

                                                                    252 257 252 257

                                                                    • 31. Re: Color flickering in sequence of identicially processed raws
                                                                      johnrellis Most Valuable Participant

                                                                      You won't see any color change if the values are all equal.

                                                                      LR shows changes whether the values are all equal or not.  Here's the original (252, 253, 253, 254), one with (100, 100, 100, 100), and one with (100, 400, 400, 200) (Exiftool order):

                                                                      Here's what they look like in Olympus Viewer 3:

                                                                      • 32. Re: Color flickering in sequence of identicially processed raws
                                                                        trshaner Adobe Community Professional & MVP

                                                                        I agree it appears Olympus 3 Viewer does not use that EXIF field Black Level readings. It could be that its raw converter uses the actual Black Row pixel sensor data and a more complex algorithm instead of the EXIF data. It has to set the Black Level somehow or the image will have much higher noise.

                                                                         

                                                                        Canon cameras ( and others?) use a dark masked pixel area at the end of each sensor pixel row (Dark Row). The Black Level offset varies with temperature and the sensor die will have hot-spots. By determining the black level for each row it provides more accurate Black Level calculation. It may be that LR and DCRAW don't use the Dark Row Black Level data with ORF files and simply apply the global EXIF data Black Level values. Just a guess out of a myriad of other possible causes, but Olympus Viewer 3 is most likely using proprietary Dark Row Black Level data embedded in the ORF file.

                                                                         

                                                                        If interested some more information I found here: Offset/black level in Canon raw files - Open Photography Forums

                                                                        • 33. Re: Color flickering in sequence of identicially processed raws
                                                                          johnrellis Most Valuable Participant

                                                                          Here's another example of someone with an Olympus E-M5 seeing MakerNote black levels differing by 1 (254, 255, 255, 254) causing an unexpected color shift: https://discuss.pixls.us/t/wrong-default-black-point-settings-for-olympus-e-m5/1988 . This is with Darktable, which apparently uses the Rawspeed library for most cameras and Dcraw for the others.

                                                                          • 34. Re: Color flickering in sequence of identicially processed raws
                                                                            johnrellis Most Valuable Participant

                                                                            And in that example too, LR shows the color shift, while Olympus Viewer 3 does not.

                                                                            • 35. Re: Color flickering in sequence of identicially processed raws
                                                                              trshaner Adobe Community Professional & MVP

                                                                              Looks like the exact same issue. It's an under-exposed image with a +1 difference in the G and G2 Black Levels. Changing it to 254 All in RawDigger and then opening the TIFF export in LR removes the Green channel WB non-linearity. Using the WB Eydropper on the sweater and gray background produce different results with the ORF file and virtually identical with the RawDigger corrected TIFF.

                                                                               

                                                                              The below images were asdjusted using the LR WB Eyedropper on the sweater.

                                                                               

                                                                              Mairi_Troiseme ORF_RawDigger Black Level 254 All.jpg

                                                                              • 36. Re: Color flickering in sequence of identicially processed raws
                                                                                gmyny Level 1

                                                                                Thanks much all of you for digging into this and sorry that I missed most of the show.

                                                                                 

                                                                                Since the sequence that triggered this question was during the night at pretty stable conditions, there are no pictures that wouldn't require an exposure correction. I actually did a timelapse the following night, with manual WB setting in the camera, resulting in the same color flickering.

                                                                                 

                                                                                I remember that I had a similar flickering once before, also during night with some faint Auroras - so also with a significant over exposure in post processing. If I reverse this statement: All timelapse sequences without exposure correction do not show a visible color flickering.

                                                                                 

                                                                                I'll try changing the Black Levels in the EXIF data, but it looks like this should solve my problem.

                                                                                 

                                                                                Cheers

                                                                                gmyny

                                                                                • 37. Re: Color flickering in sequence of identicially processed raws
                                                                                  Bjornago Level 1

                                                                                  Having exactly the same problem with my night sequences... changing Black Levels in the EXIF data has made no difference... can be seen in both Olympus Viewer 3 and in Light room...

                                                                                   

                                                                                  Chatting to Olympus, fingers crossed for a solution...