14 Replies Latest reply on Dec 19, 2010 1:16 PM by Koray Birand

    Gamma problems

    TynanW

      Hi there . . . . .

       

      When I export stuff from AE in most formats (H.264, MPEG4 etc) the render is pale, washed out and much lighter than my AE project.

       

      I know there is a gamma issue with H.264 and quicktime, but MPEG4 (in a Quicktime wrapper) now also seems to suffer the same problem, whereas it didn't pre-Snow Leopard ?

       

       

      Anyone got any clues as to the root of the mismatch between AE and Quicktime player ?

       

       

       

      P.S. If I render out as ProRes the problem disappears ?

       

       

       

       

       

      Cheers

       

      Lee.

        • 1. Re: Gamma problems
          Mylenium Most Valuable Participant

          Please read and then follow the linky-linkies:

           

          http://kb2.adobe.com/cps/516/cpsid_51667.html

           

          For more info on ProRes Gamm handling, read this:

           

          http://blogs.adobe.com/toddkopriva/2009/12/prores-4444-colors-and-gamma-s.html

           

          Mylenium

          • 2. Re: Gamma problems
            TynanW Level 1

            Cheers Mylenium . . .

             

             

            Went through that lot, my head feels a little hotter now, might have to lie down for a few hours !

             

            The Adobe link (which then links to Apple for further torture) is fairly complex, but I followed as best I could, it speaks about images (not videos) being darker (not lighter) . . . but I took a few pointers and tried various stuff in AE (changing the colour profile settings in Project Settings . . . and playing with the Color Management in the Output Module).

             

            I managed to get the Quicktime to look as saturated (I think I changed the project settings to Apple RGB) as the project in AE . . . . but it was noticeably more yellow / less red.

             

            Damn ! You need a PHD and two degrees for this stuff . . . . all I want is to be able to render out a Quicktime movie that looks the same as my AE project !!!

             

             

             

             

            [is there an icon for a crying adult around here ?]

             

             

            Lee.

            • 3. Re: Gamma problems
              TynanW Level 1

              Just done more (much more) trawling around the internet . . . doesn't seem to be any clear answers, I simply cannot believe such a basic and fundamental problem has not been addressed !

               

              The idea that AE offers you tools of such precision for colour manipulation like Colour Finesse - that you can spend a lifetime tuning to perfection - only to see a large scale colour, saturation and contrast change imposed onto your output file !?

               

              Even more frustrating is that previously (I have just upgraded to CS4 + i7 imac + Snow Leopard) this gamma shift (if that is what it is) only really effected H.264 renders - but now MPEG4, H.264 and even Animation (set on full quality) render as washed out, low contrast and pale compared to the AE project.

               

              : (

               

              I spend my working day mostly doing audio in Logic - if when bouncing/rendering your finished piece of music Logic rendered it as dull, quiet with everything pitched down a little, people would abandon Logic in droves, I am curious why this AE problem is not occupying the top 200 enquires here on Adobe support forums !!?

               

               

               

              Is there anyone out there with a Mac who has solved this problem ?

               

               

               

               

               

              help !!

               

              Lee.

              • 4. Re: Gamma problems
                TynanW Level 1

                Bump !!

                 

                Can anyone point me towards an AfterEffects website or forum or online community of some kind where I might be able to find an answer to this problem ?

                 

                 

                Cheers for any help.

                 

                 

                 

                Lee.

                • 5. Re: Gamma problems
                  nelson1138

                  I've been having gamma issues as well. I've tried the Todd Kopriva solution for ProRes 4444, but no luck with that. Any solutions would be GREAT. I'm still trying to figure out it it is an After Effects, Quicktime, or Final Cut Pro problem.

                   

                  Kent

                  • 6. Re: Gamma problems
                    Todd_Kopriva Level 8

                    Lee,

                     

                    There are quite a few intertwined issues regarding color shifts and gamma shifts. One has to do with QuickTime and its (sometimes) mistaken use of its gamma tag for some kinds of video. For information about this, see

                    "QuickTime and gamma in non-color managed projects". The upshot is that sometimes you need to select Match Legacy After Effects QuickTime Gamma Adjustments in the Project Settings dialog box to ensure that colors that you see in the Composition panel in After Effects match what you see in QuickTime player.

                     

                    You also mentioned a difference between Snow Leopard and Mac OS before Snow Leopard. As you may know, Apple changed the gamma in Mac OS for displays from 1.8 to 2.2.

                     

                    If you want to ensure consistent colors, use color management. For step-by-step instructions for using color management, see this document.

                    • 7. Re: Gamma problems
                      TynanW Level 1

                      Thanks Todd.

                       

                       

                      I will start to trawl through those links today or (probably 'and' !) tomorrow.

                       

                      I was hoping for a "you need to press this button" kind of answer - so I am a little disappointed to see a 32 page technically dense document that needs not only to be read - but to be understood !

                       

                       

                       

                      I just make pretty pictures for people - so when told that - "CiElAb (l*a*b*) is a device-independent color space based on the CIE XYZ (1931) color model", you can imagine my poor brain heats up and tells me to go and make a cup of tea.

                       

                       

                       

                      TODD: **The upshot is that sometimes you need to select Match Legacy After Effects QuickTime Gamma Adjustments in the Project Settings dialog box to ensure that colors that you see in the Composition panel in After Effects match what you see in QuickTime player.**

                       

                      I tried this - it made no difference at all - and when I say no difference I mean absolutely no difference - and I test things very methodically.

                       

                       

                      I am shocked to be honest - I am working on such a predictable system:

                       

                      A brand new copy of Adobe CS4.

                       

                      A brand new 27" i7 iMac running 10.6.2.

                       

                      No third party plug-ins or extensions of any kind.

                       

                      A complete, virgin, clean, unadulterated system running entirely standard software.

                       

                       

                      But I can open a movie (un-converted - directly from a Canon 5Dmk2) and it looks fine when played in Quicktime player - darker and more saturated along with a colour shift towards red/yellow when brought into After Effects - pale, flat and washed out when rendered and played in Quicktime player - and then darker and more contrasty when uploaded to online places like Vimeo/YouTube.

                       

                      At no stage do you ever see two matching appearances, for instance the pre and post AE Quicktimes do not even match.

                       

                      The really shocking part is that this issue has not been - as a matter of default - addressed between Apple and Adobe, I cannot believe professional image software - for which I paid a small fortune - happily ships with such a enormous mismatch between input and output, if the look of moving images can veer so wildly just within my system I have zero confidence that I can predict what the final piece will appear like on TV or the internet, let alone take confidence in using something like ColourFinesse and even just applying curves ! What after all am I applying these things to, if I spend a while in curves pushing the yellow a little in the shadows . . . . for all I know it is already a little too yellow ?

                       

                      I have some work coming up for Greenpeace, Discovery and Hallmark - and it looks likely I will have to dig out my copy of AE7 and see if I can find a secondhand G5 somewhere - which is a pain.

                       

                      Do you know if there is any plans to resolve this - quite frankly enormous - issue ?

                       

                       

                       

                       

                       

                      Thanks for your help Tood, much appreciated.

                       

                       

                       

                      Lee.

                      • 8. Re: Gamma problems
                        Navarro Parker Level 3

                        So Tim, in a perfect world, is it best to use linear gamma as much as far as you can and then introduce log gamma at the final render step?  There's a De-Gamma plug-in for Cinema 4D that looks intriguing. And I hear all the time from the Nuke/Shake crowd that linear is the key to getting the best looking composites.

                        • 9. Re: Gamma problems
                          ChrisProsser Adobe Employee

                          On another thread the original poster here qualified that they are using the File Menu->Export... command to export.

                           

                          Without getting into technical mumbo jumbo, After Effects doesn't have much input as to what happens in that menu and can't do much to fix color problems. When you select something from the Export menu, all control other than a rendered frame are outside of the control of After Effects.

                           

                          Our effort is spent making the Render Queue export work correctly. For Render Queue based exports we control the exporting sequence and components much more carefully. This means some cool things like USB based h.264 accelerators can't be supported by the render queue. But it also means that we can more accurately control the color conversions that quicktime is doing under the hood as to avoid gamma shifts.

                           

                          -c

                          • 10. Re: Gamma problems
                            TynanW Level 1

                            **On another thread the original poster here qualified that they are using the File Menu->Export... command to export.**

                            .
                            .

                            Stop !
                            Right !
                            There !
                            .
                            .
                            The problem manifests itself whether you choose Make Movie, Export File or the Render Queue - I have dedicated hours (perhaps cumulatively days) to establishing the nature of the issue, one thing I can say with reasoned inerrancy is that it is not the choice of output route.
                            .
                            .
                            **Without getting into technical mumbo jumbo,**
                            .
                            .
                            Please do !
                            .
                            .
                            There are people, your customers who are struggling to achieve basic delivery requirements, if you can get mumbo jumbo on our ***** and it lends something to solving this problem, then please do !
                            .
                            .
                            I think it is worth the effort for those who want accurate colour workflow to engage in this issue.
                            .
                            .
                            **Our effort is spent making the Render Queue export work correctly.**
                            .
                            .
                            With this as the defined context could you offer a solution to my issue with the render output mismatch ? My own experience is that how ever you export from AE the problem exists, this seems to be a common and universal issue echoed by numerous AE users.
                            If I render a file and it is washed out or is rendered with a colour shift - then wether I use the render queue or simply use the export file route makes no difference - they will both suffer the same fate.
                            And when I say the same, I mean they are exactly the same, every pixel value is identical - (files are interrogated using difference mapping - and direct value sampling) - using export file or render queue is not the issue here.
                            .
                            .
                            If you consider that the Render Queue export works correctly - could you outline in simple terms the process of - for instance and as a good starting point on which to build further knowledge - how to import a Quicktime - and then render it out - and maintain colour accuracy for a predictable system ?
                            .
                            .
                            Any input gratefully appreciated, I am amassing a collection of information regarding the problem - none of it definitive - unfortunately the overarching indication seems to suggest that this is a somewhat intractable problem that can - at best - only be mitigated by a compromise over control of your final render.
                            .
                            .
                            **But it also means that we can more accurately control the color conversions that quicktime is doing under the hood as to avoid gamma shifts**
                            .
                            .
                            No.
                            .
                            .
                            I hate to sound so absolute, but using one render route over another does not address this issue, the problem exists - with identical colour and gamma offset whether you use one route out of AE or the other.
                            .
                            .
                            If I am wrong here, could you tell me how to avoid gamma shifts ? I can list my system (it is banally straightforward with only standard issue Mac/Adobe elements) if that is a help ?
                            • 11. Re: Gamma problems
                              ChrisProsser Adobe Employee

                              Thanks for putting in hours of time, this whole area is a pain. I need to point out that while I understand the technical issues, the guy who knows the practical issues is out due to a family emergency so I will be a poor balm.

                               

                              Where to start?

                               

                              After Effects uses RGB internally to represent color. Using the export menu has Quicktime drive the whole export process, which includes having quicktime call back into AE to render an RGB frame for quicktime to go compress. When using the render queue, AE is master, it renders a frame and then pushes it to quicktime. When we do the pushing, we get the opportunity to set some setting that sometimes help with gamma and color shifts. I'll call out what we do in the render queue path that we can't do in the Export menu path due to the underlying architecture.

                               

                              The problem with gamma and color shift almost exclusively occurs with Codecs that run in YCrCb. This includes all ProRes (I'm lumping in ProRes 4444 even though it can do RGBA) and even PhotoJPEG if you aren't at highest quality (I think above 75%). These codecs are often defined to be in either the Rec601 or Rec709 color space (http://en.wikipedia.org/wiki/Rec._709). My guess is PhotoJPEG is sRGB, but that is a wild guess. Quicktime makes certain assumptions about the RGB color buffers it gets. These assumptions are different in Snow Leopard, Leopard, and Windows.

                               

                              Since the codec is expecting color to be represented a certain way, the assumptions and associated color conversions when going from RGB to YCrCb is where things go awry.

                               

                              By default Quicktime assumes RGB buffers are in a monitor color space. I believe this means sRGB (http://en.wikipedia.org/wiki/SRGB) with a 1.8 gamma on Leopard (technically not sRGB, but what can you do), sRGB with 2.2 on Leopard, and sRGB with 2.2 on Windows. Since Rec709 and sRGB have the same color primaries, any incorrect assumptions here only appear as brightness/gamma changes.

                               

                              The idealized black box export path for the Export Menu is:

                              AE->RGB (untagged)---Color Conversion in Quicktime--->YCrCb--->compressor.

                               

                              In CS4 (maybe CS3?) we added the ability to tag the RGB buffer with the gamma. AE also will request RGB frames from Quicktime with the GAMA tag applied when importing. This is controlled with the "Legacy Quicktime Gamma" preference. I believe the export half of this will only affect the Render Queue and not the Export menu.

                               

                              The intent behind using the gama tag was to make the RGB to YCrCb conversion more predictable. Alas, the tag ends up in the resultant movie and can influence other consumers of the file (like FCP). Further confounding things. This is why we added the Legacy Quicktime Gamma checkbox, we could not make every single test workflow work. The Legacy workflow does not add the tag.

                               

                              Lastly, not only does each product on the market talk to Quicktime differently (i.e. some use RGB, some use YCrCb buffers directly, some obey the gama tag for color conversions, some don't), but some codecs themselves implement color conversions. Also, some 3rd party codecs install their own color conversion routines that compete with the Apple defined ones and will cause different RGB to YCrCb conversions even when NOT using their codec. Also, Quicktime makes different guesses based on which OS you are running.

                               

                              Since a couple of ProRes codecs were shipped after CS4, they don't work correctly out of the box. They need some custom settings (http://blogs.adobe.com/toddkopriva/2009/12/prores-4444-colors-and-gamma-s.html).

                               

                              Also, pixel dimension will also affect the color conversions as some codec use non-HD stuff as Rec601, not Rec709 which leads to another source of shifts.

                               

                              In the end the RGB to YCrCb color conversions are influenced by:

                              1) Your export path (Export Menu, vs Render Queue)

                              2) The operating system you are running

                              3) Which codecs are installed on the system (even if you aren't using them)

                              4) The setting of the Legacy Quicktime Gamma Setting

                              5) The particular destination application.

                              6) The pixel dimension of the footage.

                               

                              I also feel the need to point out that while ProRes may be in Rec709 (which nominally has a gamma of 2.2), Final Cut Pro will correctly convert YCrCb Rec709 to Monitor RGB (Gamma of 1.8 on Pre Snow Leopard) for display. By default After Effects does no display color correction. When importing a file that starts in YCrCb Rec709 we convert it to RGB Rec709 (Gamma 2.2) if AE does the decoding. This means something like a DV file will look different in AE and FCP on a mac with 1.8 display gamma. If you use an external preview device, I do not believe you will see any difference, but things will look different on the mac monitor.

                               

                              I'm hungry enough now that I'm just going to make more errors than I already have.

                               

                              I hope this helps someone.

                               

                              --chris

                              • 12. Re: Gamma problems
                                TynanW Level 1

                                Cheers for your reply Chris, very much appreciated !!

                                 

                                I will go through it very carefully all tomorrow, hopefully it offers some healthy direction for those of us struggling with this.

                                 

                                 

                                 

                                Thanks once more for the help.

                                 

                                 

                                 

                                 

                                 

                                 

                                 

                                 

                                Lee

                                 

                                 

                                P.S. Don't be shocked to hear a few (hundred) questions coming back at ya' !

                                • 13. Re: Gamma problems
                                  ChrisProsser Adobe Employee

                                  I won't be shocked, but I'm pretty slammed the next couple of weeks, so my posts will probably be shorter.

                                   

                                  I just remembered another salient fact. For certain codecs (not ProRes) that are in a .mov file, we will decompress the video frames internally, including the YCrCb to RGB conversion, but on export we will use quicktime to perform the RGB to YCrCb. This was the original reason for adding the GAMA atom.

                                   

                                  My parting shot. If you are running Leopard, and have a piece of footage in a YCrCb codec (i.e. ProRes) you will get different on screen results when you import that into FCP and AE because FCP is applying display compensation. You will need to set your project working space to Rec709 to see the same thing.

                                   

                                   

                                  -c

                                  • 14. Re: Gamma problems
                                    Koray Birand

                                    TynanW I wonder if you have solved this problem because I am struggling with the same problem... And I have been posting on the same forum :

                                    http://forums.adobe.com/message/3349744#3349744

                                     

                                    But can not get a clear answer...

                                     

                                    Koray