19 Replies Latest reply on Jul 26, 2009 1:45 AM by T27

    HD Workflow - Need Some Clarification Please

    T27 Level 1

      Apologies for asking another question so soon, but I need some help with my import/export workflow please.

       

      So I've just upgraded to AE CS4 from 6.5 Pro.

       

      I'm doing a show where I'm using XDCAM EX footage with cgi shots and elements, all comped in AE.  The shots will be handed to the editor, who uses FCP, as Quicktime files, I think.  This is where I need some help.

       

      I have read and re-read (several times)  the 'Color Management Workflow in Adobe After Effects CS4' pdf, specifically the 'High Definition Video Workflow' section. I am using a calibrated monitor (regularly calibrated and checked with a Spyder 3 Elite).

       

      I get setting up the AE project to be colour managed to the HDTV (Rec. 709) working space, and I likewise understand using Interpret Footage to colour manage assets that are imported.

       

      Where I'm falling down is outputting the finished shots to QT movies, specifically on two points: the apparent gamma of the output (as I see it in the QT Player), and the data-rate.

       

      I understand setting the output to have the same profile in the Output Module of the Render Queue.  But from there I'm stumped, because the QT outputs don't visually match the AE comp, with almost any codec I try.  I understand there are QT codec issues when outputting from AE - I was reading about those the other day when I was hitting the same problem in AE 6.5.  When I open the QT movie on the desktop (Windows XP Pro) it's invariably alot lighter, lower contrast.  Along with the other issue that the data-rate of the QT output from AE is anything up to 8 times greater than I need.

       

      I tried rendering Animation and PNG movies from AE, then feeding those through the Media Encoder, outputting via H.264 to get an MP4 with the data-rate around 50mbits/sec.  That works on that, but again the movie is pretty washed out and low contrast.

       

      I guess this is pretty fundamental stuff, but I've hit a brick wall over what to do.  I'd appreciate any advice on the workflow to output the finished comps to hand to the editor, as a movie file, QT or whatever.  And obviously knowing that what I see in my AE comp is what he will see on his FCP suite.

       

      Julian.

        • 1. Re: HD Workflow - Need Some Clarification Please
          Mylenium Most Valuable Participant

          Data rates for intermediate files should really not be any concern. Harrddrives are dirt cheap, so simply add anotehr one. If they are, then consider buying a commercial CoDec such as Cineform or Sheer. You may also use Avid DNxHD, which is similar to ProRes and free. Regarding the Gamma issues, you may wish to assign your monitor profile as the comp preview profile. That should allow you to match things more closely. It's not perfect, though, as QT on PC is not color profile aware (it is in part on Mac). Also always check the QT's advanced information/ movie properties and turn of transparency blending. This is known to cause massive shifts in lightness.

           

          Mylenium

          • 2. Re: HD Workflow - Need Some Clarification Please
            T27 Level 1

            It's actually the data-rate of the final file which is causing problems, as well as gamma - intermediates we use are not an issue, as per hard-drive space, etc.  But the finals only need to be around the 50 mbits/sec mark, as per the camera footage.  Which again comes down to which is the best delivery format and codec for the finals which go to the editor?

             

            If my monitor profile is used for the Comp preview profile, then by definition I am no longer working to the HDTV (Rec. 709) colour space ..... or have I misunderstood how that works?

             

            Sorry if I'm being thick!

             

            Julian.

            • 3. Re: HD Workflow - Need Some Clarification Please
              T27 Level 1

              On the profile idea, in fact my monitor profile is visually very close to HDTV (Rec. 709) anyway - my monitor profile seems to be a tad redder, but subtle.

               

              So it's down to what delivery format and codec won't screw the gamma, and has a data rate more like the XDCAM EX footage.

               

              Julian.

              • 4. Re: HD Workflow - Need Some Clarification Please
                T27 Level 1

                OK - I seem to be getting good results with Avid's DNxHD.  Pretty happy with that so far.

                 

                Cineform is too expensive for me to look at right now.

                 

                Sheer is curious.  Got the trial version and exported a clip using the auto-everything setting.  Got a file 15 times larger than the original.  Those guys must have a different definition of the word 'compression'!

                 

                Also, the gamma is off, compared to DNxHD - noticeably lighter, like QT codec issues.

                 

                But, I'll play with it some more.

                 

                Thanks for the ideas!

                 

                Julian.

                • 5. Re: HD Workflow - Need Some Clarification Please
                  Mylenium Most Valuable Participant

                  If my monitor profile is used for the Comp preview profile, then by definition I am no longer working to the HDTV (Rec. 709) colour space ..... or have I misunderstood how that works?

                   

                  Actually the profile would tell your footage (to which you assigned a Rec. 709 profile) to more correctly transform back to full (calibrated) sRGB. Of course I agree - on a halfway decent monitor the differnce is probably neglectible.

                   

                  But the finals only need to be around the 50 mbits/sec mark, as per the camera footage.

                  Mmh, well, you should have enough options for that by using AE's native H.264 export or Adobe Media Encoder. They both calculate the final data rate based on your input. Generally you should prefer the native exporters over Quicktime, anyway. They have better performance and a void a number of the issues you may be seeing. It's easy to wrap such data then into a QT file by using SUPER©, MPEG Streamclip and otehr tools. Even opening the plain H.264 in QT Pro and re-exporting it using Fast Export option will do that.

                   

                  Mylenium

                  • 6. Re: HD Workflow - Need Some Clarification Please
                    Mylenium Most Valuable Participant

                    Sheer is curious.  Got the trial version and exported a clip using the auto-everything setting.  Got a file 15 times larger than the original.  Those guys must have a different definition of the word 'compression'!

                     

                    Actually they are compressing at 12bit and 14bit color depths, which always results in larger files. You should be able to tweak these settings in the CoDec's options however. By default it will always use 4:4:4 HQ YUV, I believe, but for footage that started out compressed such as yours, 4:2:2 or other settings might be sufficient and reduce file sizes considerably.

                     

                    Mylenium

                    • 7. Re: HD Workflow - Need Some Clarification Please
                      Adolfo Rozenfeld Adobe Employee

                      That is correct.

                      XDCAM is a wildly compressed format. 35 Mbps with 4:2:0 color sampling.

                      Sheer and other lossless codecs are almost miraculous in that they provide a file size savings of about 50 per cent compared with a fully uncompressed file, and with exactly the same quality (or higher, since Sheer can be 10 bit YUV/16 bit RGB). But half of uncompressed is still many times more than XDCAM. No mystery there.

                       

                      The Avid codec sounds like a very good idea if data rate is a concern.

                       

                      I don't agree about assigning the monitor profile anywhere in AE's color management settings, though.

                      AE already does that for you when you enable display color management, in the view options, provided the monitor profile is correctly assigned at the OS level.

                       

                      I personally would keep the project set as REC 709.

                      • 8. Re: HD Workflow - Need Some Clarification Please
                        T27 Level 1

                        Well, despite your suggestions on using AE's native H.264, which is giving good file sizes and imports back into AE without problem, Quicktime itself will not open it.  Neither will MPEG Streamclip.

                         

                        But you're suggesting that QT should open native AE mp4 without problem?  Or do I need the QT MPEG player?  Though again, I thought that was just for MPEG2 - the plain QT player does list mp4 in its standard file formats.

                         

                        Much as I've made alot of headway with this, and all this help has been great, it is endlessly frustrating to keep hitting walls like this.  So should native H.264 work with QT, and those other tools?

                         

                        Julian.

                        • 9. Re: HD Workflow - Need Some Clarification Please
                          Adolfo Rozenfeld Adobe Employee

                          Quicktime absolutely supports the MP4 container, and in addition, it supports the H264 codec in a MOV container.

                          You don't need the additional QT MPEG decoder at all. It's in Appe's best interest that QT opens standard flavors of MPEG-4/AVC/H264, since they're building a movie distribution business around it

                          MPEG Streamclip should also be able to open these files.

                          Do not pick the Blu-Ray encoding presets, for example, as these make use of H264 profiles/levels which may not be supported directly by Quicktime.

                          • 10. Re: HD Workflow - Need Some Clarification Please
                            T27 Level 1

                            Ah, the Blu-Ray is what I did use, so there's that one explained!

                             

                            However, I'm working with HD footage, 1080i, so I cannot use the straight H.264 ..... or can I?  I just tried in fact, but it generated an error, or warning, that I have an invalid framesize/framerate for this Level.

                             

                            But then these are delivery codecs aren't they?

                             

                            In truth I'm trying to find a good balance for an intermediate, hence the Avid DNxHD, which is still looking like the best contender for this show.

                             

                            Julian.

                            • 11. Re: HD Workflow - Need Some Clarification Please
                              Adolfo Rozenfeld Adobe Employee

                              Yes, Julian.

                              I was not suggesting a workflow. Just explaining why the resulting H264 file didn't open in Quicktime player.

                              Yes, H264 is a delivery codec. It's really good at low data rates, but it's also very demanding in terms of performance.

                              • 12. Re: HD Workflow - Need Some Clarification Please
                                Mylenium Most Valuable Participant

                                T27 wrote:

                                 

                                I just tried in fact, but it generated an error, or warning, that I have an invalid framesize/framerate for this Level.

                                 

                                But then these are delivery codecs aren't they?

                                 

                                Yes, of course they are. Nonetheless, with decent data rates they can be (ab)used as intermediates, if you're tight on disk space. Not glorious, but works and except for the performance required to decode it, not much different from working with otehr compressed formats. Regarding the profile levels: Simpyl choose a differnt one. As long as you are working locally, there's no harm in using whatever you see fit. The x.0 Levels are the most rigid ones, adhering to a sert of fixed resolutions, pixel ratios and framerates, but the x.1 and x.2 have looser specs and may allow you to use your own stuff. Anyway, if you like DNxHD, just stick with that.

                                 

                                Mylenium

                                • 13. Re: HD Workflow - Need Some Clarification Please
                                  T27 Level 1

                                  Thanks for the additional comments and ideas.

                                   

                                  In terms of profile, as I mentioned, the intermediates will be going off to the editor (off-site) for editing on his FCP suite.  I'll stick to working in the prescribed HDTV (Rec. 709) space, and I've given him all the detail I can of how I'm working, but crucially I haven't stood at his edit-suite to see how the stuff looks.

                                   

                                  But, he's no newbie, so I'm hoping it'll be OK.

                                   

                                  Blasted shame about the H.264 BluRay not being compatible with QT - I thought I had this whole issue beat until that came along.  It's not a matter of disk space here, but the editor is mithering that the intermediates we are testing, in either PNG, or Animation, and a couple of others, are choking his playback.

                                   

                                  I'm not sure what to say about that, other than to go down this road of trying to squish these intermediates as low as possible without wrecking the look, hence finding that the DNxHD is providing the best solution.  If it had been a SD project, I might well have tried the native H.264, even over our normal PNG workflow.

                                   

                                  So what will play the native H.264 BluRay compressed output?

                                   

                                  Julian.

                                  • 14. Re: HD Workflow - Need Some Clarification Please
                                    Adolfo Rozenfeld Adobe Employee

                                    Yes, the Quicktime PNG and Animation codecs are not optimized for real time playback. They will play back fine once rendered (ie, converted to whatever the FCP sequence is using as native codec).

                                    The Quicktime PhotoJPEG codec at 100 per cent quality is only a bit lossy, and it would certainly fall within "broadcast quality". It's very light to decode - FCP will chew that happily, since it's a codec it supports directly. It will probably be 2x the size of H264 for Blu-Ray, but higher quality and with better performance for editing.

                                    I don't think FCP will be happy with the Avid codec.

                                    • 15. Re: HD Workflow - Need Some Clarification Please
                                      T27 Level 1

                                      Quicktime PhotoJPEG I would use but I hit that gamma issue with it (output is too light) originally - even with the colour managed workflow, the output (viewed in Quicktime) is considerably lighter than when working in AE.

                                       

                                      Though I recall your PDFs and webpages on this do say that the RGB codecs will suffer the gamma issue.

                                       

                                      If the Avid codec is questionable in FCP - and the editor still has not come back to me on any of this - then I'm almost back to square one and my original question.

                                       

                                      Julian.

                                      • 16. Re: HD Workflow - Need Some Clarification Please
                                        Adolfo Rozenfeld Adobe Employee

                                        T27: Many other codecs you mentioned were RGB as well (PNG, Animation).

                                        I understand Final Cut Pro 6.0x has a preference item to set gamma in imported RGB material. It's in the "editing" tab in FCP's User Preferences.

                                        • 17. Re: HD Workflow - Need Some Clarification Please
                                          T27 Level 1

                                          Thanks - I imagine the editor will know about that, but I'll try to pin him down on this stuff in the coming week.

                                           

                                          PhotoJPEG, with the colour-managed workflow, may be a reasonable option, especially as you say FCP may not handle that Avid codec too well.

                                           

                                          The tone/gamma is still a little bright with lower contrast than in AE, but possibly acceptable.

                                           

                                          Julian.

                                          • 18. Re: HD Workflow - Need Some Clarification Please
                                            Adolfo Rozenfeld Adobe Employee

                                            Have you tried enabling "Final Cut Studio color compatibility" in Quicktime preferences? That should change the gamma handling behavior in QT player.

                                            Don't expect the FCP guy to be aware of the RGB gamma setting in FCP - it's a rather esoteric setting, so you may want to let him know.

                                            • 19. Re: HD Workflow - Need Some Clarification Please
                                              T27 Level 1

                                              I think that switch for 'Final Cut Studio color compatibility' is a Mac thing - there's no sign of it in the Windows version.  I guess that this setting doesn't have any direct relevance to Windows though, does it?

                                               

                                              Thanks - I'll let him know for absolute certain about that gamma setting in FCP.

                                               

                                              Julian.