8 Replies Latest reply on Jun 30, 2009 11:13 AM by mr. ichybob

    Which is best for green screen, Apple ProRes or Cineform?

    Federated Films

      I have 720p HDV material shot on green screen and wondering whether Apple's ProRes or Cineform's codec would be best for compositing it in After Effects?


      My cameraman has shown me the material perfectly composited in Cineform using Premiere CS3 on WinVista. But when I try it using ProRes in After Effects CS4 using KeyLight I never get rid of the fringing around the hair of the person in the shot.


      So ProRes and Cineform both claim to use 10bit 4:2:2 but has anyone had much experience using both codecs with greenscree?



        • 1. Re: Which is best for green screen, Apple ProRes or Cineform?
          Adolfo Rozenfeld Adobe Employee


          I am afraid your choice of codec will not improve quality (and keying) once the footage is already shot.

          Converting HDV to either ProRes or Cineform can give you better performance and a more responsive feel, because you won't be dealing with interframe compression.

          But the MPEG-2 compression (used in HDV) and the reduced color sampling already ocurred when you recorded the material. The 4:2:0 color samplig scheme in HDV is really bad for keying, since it throws away a lot of color information that is needed for pulling a clean matte.

          There are some third party keyers (like DV Matte Pro), which specialize in disguising the flaws you get when you key low color sampled material like DV or HDV.

          Some HDV cameras have an uncompressed HD-SDI output that bypasses HDV compression/color sampling completely. In this case, yes, recording (with a capture card) to ProRes or Cineform would make a world of difference compared to HDV.

          • 2. Re: Which is best for green screen, Apple ProRes or Cineform?

            What codec was it originally shot in?  HDV can be pretty painful to composite.  Also, there are a lot of different flavors of pro-res, but it can totally work for greenscreen.  Maybe get your cameraman to render a tiff sequence with an alpha channel for you if he's already got it working perfectly.

            • 3. Re: Which is best for green screen, Apple ProRes or Cineform?
              Mylenium Most Valuable Participant

              As pointed out, if it is HDV, then you already have lost a lot of information. I would not put too much face value on what your camera operator showed you on his system. The thing is, since Premiere is YUV native and operates on HDV native, it has a different way of dealing with such issues, where on the other hand everything that comes into AE is blown up back to full RGB. Furthermore, the way how AE deals with fields may give a wrong impression, which is anotehr thing that works better/ differntly in Premiere. Also you should not over-simplify the process. Rarely ever does simply throwing on a keyer produce good results from the getgo. For compressed footage, you should make it a point to remove the block artifacts first. This can be done with tools like Magic Bullet Frames, SmoothKit or a bunch of manual techniques. Likewise, noise may need to be smoothed out or fields removed. All depends. Of course do not forget, that nobody really cares how a matte is generated - duplicate the layer, ramp up contrast or whatever and key that. You can always recombine the result by using matte modes and such. Specific to Keylight: reduce the matte range by adjusting the black and white points, then use the bias and rollback settings. If you force the effect to only operate on specific ranges, the result is usually good, but if you allow it to stay at the full range, it tries to be a bit too smart and actually messes up...



              • 4. Re: Which is best for green screen, Apple ProRes or Cineform?
                Rick Gerard Adobe Community Professional & MVP

                I have used all kinds of techniques for smoothing out compressed footage that has to be keyed. The first think you need to do is to fix problems caused by the reduced color sampling used in HDV, AVCHD or DV footage. The problem is that there is no one click solution. I can give you a basic idea that seems to work fairly well for some situations working strictly inside AE without any third party plug-ins.


                The luminance value (black and white information) is what gives you the apparent sharpness and you end up with a unique value for each pixel in the image. The color values are blocks of color that are 1/2 or even 1/4 the resolution of the luminance. A solution is to blur the color using box or gaussian blur then add back in the sharpness of the luminance. The simplest way do to this, which helps immensely, is to duplicate your layer, blur the layer, change the blend mode to color, then pre-compose the pair of layers and pull your key from the pre-comp.


                When you have higher rez material, like your HDV footage you can also scale it down from 1080 to 720, to resample the color info, then render the footage, then bring add a copy of your original footage and the scaled down footage to a 1080 comp, place the 720 footage on top, scale it to fit horizontally in the frame, then change the blend mode of the rendered and scaled version to color.


                Another technique is to use the blur and blend mode in Premiere with the original footage, then render out to a suitable codec for keying in AE.


                The third party plug-in solutions mentioned also work quite well. The rub is that the best technique and most effective results are different for each shot. The only way to get you a first rate solution is to have a sample of about 10 or 20 frames of the footage where there's a lot o motion blur or transparency (like the actor holding up a wine glass) or a shallow depth of field problem you can't seem to resolve, or a color bleed problem like a blond with curly hair shot against green instead of blue. See what I mean, there's at least three things that have to be fixed with every keying project I've ever encountered. Show me some frames and tell me what tools you have available and I'll give you a suggestion as to how I would handle the problem. The codec only becomes a problem when compression artifacts are worse than color artifacts and the rest of the problems combined.

                • 5. Re: Which is best for green screen, Apple ProRes or Cineform?
                  Federated Films Level 1

                  Thanks Mylenium & Rick, lots of great advice there.


                  I take your point about it depends on the footage. So I have uploaded 1 second of my footage to my web site, encoded from a 720p m2t file into a QuickTime Animation based codec, which I pressume will have lost none of the orginal informatin?


                  Download the clip here www.federatedfilms.com/GreenScreenTest.mov


                  You may like to know it was shot using a JVC GYHD201 camera using a Datavideo CKL-100 Chromakey system and shot progressive.




                  Software wise, I have Adobe Production Studio CS4 so Premiere Pro and After Effects as well as Final Cut Studio 2



                  Thanks for your help :-)





                  • 6. Re: Which is best for green screen, Apple ProRes or Cineform?
                    Rick Gerard Adobe Community Professional & MVP

                    The blockiness is caused by the color sampling artifacts of HDV video. There are a couple of ways to fix this in AE.


                    The first is to pre-blur the matte in Keylight. A value of 1.2 to 1.8 is probably sufficient. The second is demonstrated in this image. The footage is duplicated, top layer blurred using either box blur or gaussian blur, and then the top layer's blend mode is changed to color. The image below is viewed at 1600%. You can clearly see the 4 pixel blocks of color that are repaired using the blur/blend technique.



                    A little tweaking in Keylight and you should have a decent key. Here's a sample CS4 project showing both techniques.

                    • 7. Re: Which is best for green screen, Apple ProRes or Cineform?
                      Federated Films Level 1

                      Thanks, that is very kind of you Rick to go to all of that work.


                      There is still some noise around the top of the hair which is where I was strugling before but I will try playing with Mylenium and your suggestions and see if I can get it perfect!



                      • 8. Re: Which is best for green screen, Apple ProRes or Cineform?
                        mr. ichybob Level 1

                        also, besides the excellent advice given, you can deal with problematic areas like the hair with a completly serperate key, (duplicate layer and mask).