6 Replies Latest reply on Mar 26, 2013 9:10 AM by Fuzzy Barsik

    To Transcode or not to Transcode - That is the question

    Pierre Devereux Level 2

      Hi,

       

      I have been reading some of the discussions on the forum this morning, and I noticed that there is another gap in this workflow and knowledge of files and codecs of mine.

       

      We film with our Panasonic AF101 - direct to the Samurai external recorder. The format is ProRes 422 HQ. I have spoken about this a few times before, so please forgive the redundancy.

       

      The Data rate seems to be (After a few tests) around 220MB per second. The files come out with a MOV extension (("MOov")("apch")(MC)

       

      Mainly Keying and Twixtor with some mouth animations are the effects used on the clips. All other effects are built in AE, so there are no source files and formats as such.

       

      We are ordering a Red Scarlet by the end of the week, and will use that for most of out work going forward. I have no idea how the RED works, and its recording formats, other than that it is RED RAW.

       

      My question is now twofold:

       

      1) Do I need to transcode the AF101 footage to get better quality? (In the production, I import the footage as is, and then when a intermediate render is called for, I render out the full project as TGA image sequences)

      2) Will transcoding be needed for the RED Scarlet footage?

       

      Just to put all the info in one place again - we are shooting for TV, our delivery format is MXF OP1a. The video codec must be AVC Intra codec, using the High 4:2:2 Intra profile (at 100Mbits). Character, set and external footage are either keyed, or time manipulated only. All effects are created from scratch in After Effects. Compositing is done in after effects.

       

      Once again, I do apologize for the repeat question, I just need to confirm if I should add the transcoding step in our workflow.

       

      Thank you

       

      Pierre

        • 1. Re: To Transcode or not to Transcode - That is the question
          Fuzzy Barsik Level 4
          1) Do I need to transcode the AF101 footage to get better quality?

          2) Will transcoding be needed for the RED Scarlet footage?

          When it comes to Adobe Suite, transcoding doesn't yield any extra quality. Moreover, superwhites (over bright) values may be clipped. Transcoding helps save on render time (for both RAM preview and final output). See this discussion, run your own tests and choose optimal 'quality/render time/file size' ratio (don't get over obsessed with some mathematical loss). Here is some comparison for my rig.

          1 person found this helpful
          • 2. Re: To Transcode or not to Transcode - That is the question
            Rick Gerard Adobe Community Professional & MVP

            Transcoding only helps if you have source footage that is highly compressed and that uses a GOP compression scheme. None of the formats you mentioned fall into that category. I transcode DSLR footage because nothing is lost or clipped and I get better results with the higher bit rate data when I push on it for color correction, keying, and anything else that involves changing or calculating from the original pixels. I always transcode highly compressed AVCHD footage. I never transcode from one 8 bit codec to another 8 bit codec. There's no point. I only transcode from 8 bit to 10 bit or higher.

             

            If you have a high data rate source file like your ProRez recordings you or any other high bit rate source footage you are not going to get any more out of a transcode than possibly a little faster decode from a format that requires a little less processing power to turn compressed data into the uncompressed data required for compositing.

             

            Fuzzy and I went around on the advantages of transcoding and I'll agree there are times when overbrights can be clipped, but if your source footage is from a DSLR or from an AVCHD camera, there are no overbrights in the footage to begin with. You've got a highly compressed 8 bit source with 256 distinct single values for red green and blue. That's all there is. The GoPro Cineform RAW has overbright info that can be recovered using the cineform codec so a direct transcode to another 8 bit codec is going to clip that info.

             

            There are a dozen or so workflows for working with RED and other high bit rate, high bit depth source files. I'd look at them and see what fits into your production workflow with the least amount of fussing and least amount of trouble. What ever you do, don't destroy the camera original files if you decide to transcode anything. You always want to keep them in case there's a problem you can't solve in your normal workflow.

             

            One more piece of advice. Duplicate your footage at the end of each production day. It's SOP (standard operating procedure) on every project that I produce to take the data from the days shoot, duplicate it on a separate hard drive, check the duplicate for errors, and then lock the original footage away in a safe place. We always work off duplicates. That procedure has saved me several reshoots in the last 30 years. I've never lost a single shot I had responsibility for.

            1 person found this helpful
            • 3. Re: To Transcode or not to Transcode - That is the question
              Pierre Devereux Level 2

              Hi Rick and Fuzzy,

               

              Thanks again for your replies. I keep going ove rthe same questions over and over again, but I feel that I gain a lot out of each round!

               

              Fuzzy,

               

              Thanks for the links. I have browsed them already, but will spend some more time reading them properly

               

              Rick,

               

              Thank you for your feedback as well. I will do the RED studying and see whats what. (now we have the camera, I am scouring the RED forums to try and decide what Lenses to buy - Will the madness never end?!?).

               

              I take all my daily footagem and I place it in a "Library" folder. The part of the footage that we need, I render out as the TGA file, and that is the "Source" we use for the rest of the pipeline. That way, as you suggest, our original footage always remains untouched. We then do an offsite backup of the library folder along with the edited versions. From my IT background, Backups are about the only step in the workflow I actually undserstand!

               

              You guys have a great day, I am off to study:

               

              1) Lenses

              2) Red Footage aquisition formats and workflows

              3) AE 3D and the workings thereof (Graph editor specifically)

               

              so....it promises to be a quiet afternoon!

               

              Pierre

              • 4. Re: To Transcode or not to Transcode - That is the question
                Fuzzy Barsik Level 4
                if your source footage is from a DSLR or from an AVCHD camera, there are no overbrights in the footage to begin with.

                Quite contrary: an AVC footage can contain surprisingly a lot of amount of data in superwhites. Here is an example I recently posted in PrPro Forum. A frame from a source AVCHD footage with superwhites up to 130 IRE:

                Data in Super Whites. 01.jpg

                The same frame with corrected contrast:

                Data in Super Whites. 02.jpg

                • 5. Re: To Transcode or not to Transcode - That is the question
                  Rick Gerard Adobe Community Professional & MVP

                  Don't want to start an argument here but it's the interpretation of the data in the frame by the playback codec that apparently looses the detail. Properly transcoded these values are not clipped. There are only 8 bits in AVCHD footage. No more. It's just the gamma curve that hides the apparently missing detail. You can see the same with Raw, Red Raw, even ProRez even in 10 bit or better codecs. The standard gamma curve is set by the codec. Working at a higher bit depth just gives you more room.

                   

                  My main point is that Transcoding, done properly, will give you an easier to work with product and a more efficient workflow. All modern editing and compositing systems do a fairly good job at allowing manipulation of the original 8bit data in higher bit depth projects. Properly transcoding footage from a lossy codec to a lossless format will not destroy any information in the original footage. Simply dropping the footage in the top frame into any NLE or Compositing app and rendering out a new copy won't prevent data loss. Using tools specifically designed for lossless transcoding will.

                   

                  I also said, never throw away the original footage because there may come a time when you need to retreive all of the data that was originally there.

                  • 6. Re: To Transcode or not to Transcode - That is the question
                    Fuzzy Barsik Level 4

                    Rick, I don't see the use of arguing for the sake of arguing at all. I simply share what I learnt so far: H.264 does allow to store some data in superwhites. Not only original AVC footages contain them, transcoding to H.264 or AVC-Intra also provides the same result. Here is the results of transcoding CinemaDNG sequence within Adobe Suite. Since the original format is RAW, there is a bunch of superwhites after preliminary tweaking in CameraRAW:

                    Data in Super Whites. 03.jpg

                    Superwhites are still there after transcoding to H.264 100 Mbps or AVC-Intra:

                    Data in Super Whites. 04.jpg

                    Here is the 'Difference' test between original CinemaDNG and transcoded to AVC-Intra footage:

                    CinemaDNG vs AVC-Intra 100.jpg

                    The same test for CinemaDNG vs TGA:

                    CinemaDNG vs TGA.jpg

                    The only format that allows me to get completely black solid in linearised 32 bit working space is 32 bit OpenEXR.

                     

                    My point about transcoding - it's just a tool. Sometimes extremely useful, sometimes unnecessary. Quote from you, 'Planning is the key'.