15 Replies Latest reply on Dec 5, 2009 8:37 PM by Jim_Simon

    Why are my blacks at 16 instead of 0 out of 255

    remainz123

      HI

       

      I have just mastered my 1080p AE project to 32bit Avid DNxHD via media encoder.

      But my blacks at 16 instead of 0 out of 255 color range.

       

      Am I stuck with this or can I fix it?

      Avid DNxHD is really the best output option for me due to file size even though its very slow to render.

       

      thanks

      remainz

        • 1. Re: Why are my blacks at 16 instead of 0 out of 255
          Mylenium Most Valuable Participant

          Check the CoDec's specific options. Set the Color levels to RGB, not Rec. 709. make sure to install the latest version of the CoDecs, older versions are always Rec. 709.

           

          Mylenium

          • 2. Re: Why are my blacks at 16 instead of 0 out of 255
            remainz123 Level 1

            ok thanks that bit was gobbeldegook to me

             

            remainz

            • 3. Re: Why are my blacks at 16 instead of 0 out of 255
              remainz123 Level 1

              HI

              I set Color levels to RGB and there is no difference. Black is still at 16 out of 255.  Im guessing that its adobe media enoder misrendering it as it doesnt do it when I export to JPG2000 codec and then to Avid DNxHD within quicktime export itself. This is a very long way around. It takes my quad core pc 9 hours to render 6 minutes of JPG2000.

               

              So Adobe Media encoder is misreading the gamma levels? I think I read of a similar bug in quicktime H264 export.

               

              Back to square one then , HD export formats for mastering and post production are very badly supported by adobe!

              • 4. Re: Why are my blacks at 16 instead of 0 out of 255
                Adolfo Rozenfeld Adobe Employee

                remainz123:

                Many codecs compress the color values to the 16-235 range, since that's the legal range for broadcast video.

                Many of these codecs have an option to use 16-235 or full range (0-255). I don't have the Avid codec at hand now, but the RGB option should work at full range.

                In any case, this range compression is performed by the codec itself, not by the host application.

                 

                 

                 

                So Adobe Media encoder is misreading the gamma levels?

                 

                I don't know, I'll try to test this later.

                FWIW, I didn't and personally wouldn't recommend Avid codecs in a non-Avid workflow.

                • 5. Re: Why are my blacks at 16 instead of 0 out of 255
                  remainz123 Level 1

                  I just realised that I can export using quicktime from the File - Export menu and yes it doesnt set the blacks to 16. So it is not the codec but media encoder exporting badly. ME also has a pop up window for the codec settings that hides the fine options as you cannot resize it.

                   

                  So if you need to make an AVID DNxHD file you have to use file export and not media encoder.

                  • 6. Re: Why are my blacks at 16 instead of 0 out of 255
                    remainz123 Level 1
                    FWIW, I didn't and personally wouldn't recommend Avid codecs in a non-Avid workflow.

                    That maybe the case but adobe doesn't offer any suitable alternative for mastering HD for online (hard drive or other media) backup. Not surprisingly I don't have an HDCAM SR or similar machine here and I wouldn't only want tape copy masters even if I did had one. All the places I know that have HDCAM SR equipment are not using adobe premiere or AE either, so that's catch 22.

                     

                    I think you need to rethink why you have no supported HD online post production formats except uncompressed.

                    • 7. Re: Why are my blacks at 16 instead of 0 out of 255
                      Bob Currier Level 3

                      Are you sure your post house doesn't want blacks at 16? That's the norm for Avid workflows.

                       

                      And blacks at 16 implies whites at 235, which provides "whiter-than-white" headroom for specular highlights. Mastering to a 0-255 range clips those highlights and in my experience would be considered a bug, not a feature, in an Avid-based workflow.

                       

                      Or is your post house not using Avid, but something else which is using the Avid codec? In which case, it's possible that software is not properly interpreting the coding range.

                       

                      In any mixed workflow all the pieces have to match. Make sure you know what you should be matching to before assuming that any one piece is at fault.

                      • 8. Re: Why are my blacks at 16 instead of 0 out of 255
                        remainz123 Level 1

                        Thanks for your thoughts but I think it is you that has jumped to a conclusion. I am not making work for broadcast but short film festivals and I need to have an online master that has all the color information in it. I couldn't care less that broadcast has archaic color ranges.

                         

                        As mentioned above Adobe does not have any suitable HD mastering formats. That is the issus at hand. Avid seems to , and has been generous enough to make its codec free via a quicktime plugin.

                         

                        Also mentioned above I have shown that its adobe media encoder is incorrectly setting the blacks to 16 as quicktime avid HD export does not do this. Why would the same codec have completely different color ranges if you export it via different paths? that does not make sense.

                         

                        All that aside I have to render off each of my comps to a format to then reimport so that I can timestretch them, as precompose does not save the fps info. Another major AE flaw. So I have to find some sort of format to do this. Uncompreseed is not an option in a 15 minute project so I need a format and codec that can handle this. Adobe has none!!! So the best next options are via quicktime. JPG 2000 takes 9 hours to render 6 minutes of footage. Again not an option. So what left ???

                         

                        The list of pain in the *** issues because there is no supported HD format is as long as my arm!

                        • 9. Re: Why are my blacks at 16 instead of 0 out of 255
                          Bob Currier Level 3

                          Your goal seems to be to get your AE project out to HDCAM-SR, correct? And since you don't have a deck you're relying on someone else to do a layoff to tape, right?

                           

                          This is a very common process, so it's not like you're the first to try to get it working. Asking how others accomplish that will probably yield better results than complaining that AE can't do things that others are doing every day.

                           

                          The Avid DNxHD codecs are a perfectly fine way to accomplish this, as at higher data rates the codec is very "transparent" but saves a lot of disk space and transfer time. But do make sure that whoever is laying the footage off to tape can handle those codecs.

                           

                          The thing you seem to be rebelling against is that the Avid codecs expect to work in 16-235 mode. As the saying goes, it's not a bug, it's a feature. And it has nothing to do with "archaic color ranges" and everything to do with preserving the full range of available luma values. The SR deck expects that range and records that range to tape, so I'm not sure why you are so sure you don't want it.

                           

                          But the key to making the process work is to ensure that what you produce matches what is expected when the layoff to tape occurs. Have you tested any of your Avid-encoded footage to see if there is a problem? After all, what matters is that when you play back the SR tape that your levels are correct. What happens in between really doesn't matter.

                           

                          The reason you are seeing different results in QuickTime and AME is that the Avid codec has an RGB compatibility mode so that Avid systems can export data to applications which can't handle 16-235 ranges. In RGB mode luma is mapped to 0-255. I believe QT defaults to that mode, but you can change that option in the codec's compression settings, at least if you have QuickTime Pro. AME seems to default to 16-235 (which is a better default, IMHO).

                           

                          If you need an intermediate codec for QuickTime, try Photo JPEG at 90%. It's much faster that JPEG-2000 and gives very good results with a fair degree of compression. It's a commonly used intermediate format.

                           

                          I'm not sure what to make of your issue with precomposing not preserving the frame rate. That doesn't sound right, but there are real AE experts around here who might be able to help with that.

                          1 person found this helpful
                          • 10. Re: Why are my blacks at 16 instead of 0 out of 255
                            remainz123 Level 1

                            Thanks for your suggestion

                             

                            My bug bear is that there is no clear HD workflow. I do not specifically want HDCAM SR just the best intermediate and final formats there are and who wouldnt, so that my final project can be made into as many formats later as possible. HDCAM SR is the best HD format there is at present as far I know, except that adobe hasn't made an export format for it and no alternative. So I am probably just confusing the issue by using it as an example.

                             

                            photo JPG 90% is a compromise and not an obvious working format to swap between exports. But I will now try this as a fourth possible option.

                            I dont think that adobe is suggesting that photo jpeg is the best way to save off full HD data as a hard drive master. If so I apologize profusely

                             

                            ps..I have talked in depth about time stretching in other topics and there is no alternative but to render and then reimport. When you timestretch a precomposed comp it does not evenly distribute the frames. If I use the avid codec I then lay the incorrectly rendered sections over a black background and the black at 16 does not match the black at 0. I can change the black at 0 to match the black at 16 but Im sure youll agree that thats not the way it should be done!

                             

                            ppss. I just tried photo jpeg and it was slower than JPG2000 by 100%. So I still do not have a working HD format if Avid DNxHD cannot work in the full color range.

                            • 11. Re: Why are my blacks at 16 instead of 0 out of 255
                              Adolfo Rozenfeld Adobe Employee

                              I just realised that I can export using quicktime from the File - Export menu and yes it doesnt set the blacks to 16. So it is not the codec but media encoder exporting badly. ME also has a pop up window for the codec settings that hides the fine options as you cannot resize it.

                               

                              So if you need to make an AVID DNxHD file you have to use file export and not media encoder.

                               

                              I think the problem here is that you are exporting using the File > Export command or AME, instead of the main way for rendering files in AE, which is exporting from the Render Queue. Is there any reason why you are not doing this?

                              The File > Export > Quicktime path is really not recommended. Among a bunch of other things, features like multiprocessing, Color Management, etc won't work at all when using this command. And this is why AE CS4 shows a warning when you do that. Any options you see there will also be available from the Render Queue, and a lot more. And just the fact that you can use multiprocessing ("Render Multiple Frames Simultaneously") could make things render much faster in many cases.

                              The Adobe Media Encoder is meant to be used mainly for distribution formats, not production formats.

                              1 person found this helpful
                              • 12. Re: Why are my blacks at 16 instead of 0 out of 255
                                Adolfo Rozenfeld Adobe Employee
                                photo JPG 90% is a compromise and not an obvious working format to swap between exports. But I will now try this as a fourth possible option.

                                All of them are compromises, we tried to go through many of these in the posts above. DNxHD also has its set of pros and cons, depending of what you want it for.

                                That said (have you seen "Curb your enthusiasm" recently? ), PhotoJPEG has a devoted following. At 100 per cent it has 4:4:4 sampling and it's visually lossless. At 90 per cent, it's 4:2:2 and it may still qualify as visually lossless. It's also very fast to decode and encode, which leads us to...

                                 

                                 

                                pss. I just tried photo jpeg and it was slower than JPG2000 by 100%. So I still do not have a working HD format if Avid DNxHD cannot work in the full color range.

                                All this time you've been trying these things with File > Export > Quicktime, right? If so, now we know why it all goes so badly for you.

                                Even the slowest codec in the world should have a relatively minor cost in rendering times, compared to (say) adding Motion Blur to just one footage layer in your Comp.

                                 

                                I dont think that adobe is suggesting that photo jpeg is the best way to save off full HD data as a hard drive master. If so I apologize profusely

                                 

                                Adobe is a large company

                                Quicktime PhotoJPEG certainly is considered by most experts as "online quality". I personally (not Adobe) would indeed recommend it in many cases. And no less than Bob Currier (the father of Color Finesse) also said it's fine in this very thread.

                                There will always be something better or more suitable for a specific workflow (you wouldn't use PhotoJPEG for feature film compositing).

                                • 13. Re: Why are my blacks at 16 instead of 0 out of 255
                                  remainz123 Level 1

                                  thanks adolfo

                                   

                                  I will use the render queue again.

                                   

                                  I have got into the habbit of using AME because I do a lot of MPEG2 HD renders for playing on media servers and for one AE render queue does not allow full HD frame size options for this. Why??? I also like to keep large AE project seperate due to memory and then render them with AME instead of bringing them all into the same AE project and creating a render queue which can get very complicated if I want to change something in the original and have to completely remake the whole collection of AE projects everytime I want to sent up a few renders. Again these are all problems because there is no clear HD workflow in AE. If the render queue could do this I wouldnt be using AME and visa versa.

                                  • 14. Re: Why are my blacks at 16 instead of 0 out of 255
                                    Adolfo Rozenfeld Adobe Employee
                                    I have got into the habbit of using AME because I do a lot of MPEG2 HD renders for playing on media servers and for one AE render queue does not allow full HD frame size options for this. Why??

                                    Yes, each MPEG-2 flavor in the embedded version of AME in AE locks the parameters that would make the resulting file not legal for the chosen preset. T

                                    The MPEG-2 Blu-Ray preset will produce a file like the one you're describing. But, yes, for distribution formats like MPEG-2 the standalone version of AME is a good idea, since it offers more flexibility in encoding options (like 2 pass VBR encoding).

                                    • 15. Re: Why are my blacks at 16 instead of 0 out of 255
                                      Jim_Simon Level 8
                                      adobe doesn't offer any suitable alternative for mastering HD for online (hard drive or other media) backup.

                                       

                                      Adobe doesn't, but others do.  Both Lagarith and UT codecs work well in the Adobe suite, and both are lossless.