25 Replies Latest reply on Oct 16, 2012 5:06 PM by josephs51576386

    Exporting Master Archive Copy of Project on Mac

    DMH79 Level 2

      I know, I know. There are like 100 threads on final export and the best ways to do it for archival purposes. But honestly, I'd read one and it would say one thing and then I'd read another and it would say another. I guess that makes sense since there probably isn't the ONE best way to do anything. But coming from FCP, to export a archive copy of your final project you simply selected EXPORT>Quicktime Movie and it would do it all for you with no adjusting or tweaking of any settings necessary. It simply would save it using your sequence settings as a copy that you could use as an archive OR (and this part is important to me) you could take that file and bring it back into Compressor later if you wanted to encode this master copy into other formats (DVD, Bluray, H.264 for web, etc.) with the least amount of quality loss. Its almost as if you had exported the video directly from the original timeline. That's what I'm looking for here.

       

      All that being said, I know there is a "Match Sequence Settings" box in Adobe Media Encoder which one would think would be similar to FCP's "EXPORT>Quicktime Movie" option but I've read in various threads not to use that and it made me nervous anyway because when you select it the "quality" slider is set to 50% and greyed out and the resulting file was going to be .mpeg which I didn't know if that was the best option or not to preserve quality. My original source footage is AVCHD so I've been going back and forth editing native and editing in ProRes 422.  Since I'm on a mac (2011 27" iMac i7 with 16gb ram, AMD 6970M card etc.), I've read of a few options:

       

      1) Quicktime using the Animation Codec.

      2) The mac version of the UT video setting

      3) ProRes HQ or 422 (however I've also read that this isn't the best option if you want to re-export the footage later into other formats)

       

      Various other virtual and lossless  methods but many of which seem to be for PC users. I am a mac user.

       

      My goal is to have a master copy that will need no more editing but one I can drop into AME or even Compressor to make future DVDs, BLURAYS, High quality H.264 files for web delivery, etc.

       

      While I realize there isn't one answer probably, what are the advantages/disadvantages of the 3 methods above and is there a better way to export a file for archiving and possibly re-encoding later. File size doesn't really matter. Well, within reason. With FCP, an hour long sequence would usually be under 50gb if I remember right. So anywhere in that ballpark (or a bit higher even) is cool with me.

       

      Please feel free to describe in detail the steps to making this MASTER file of the final edit.

       

      Thank you!

        • 1. Re: Exporting Master Archive Copy of Project on Mac
          ExactImage Level 3

          When I need to store a master copy of a sequence that I know will go to other formats, including Blu-ray, DVD and H264 for the web I export to ProRes 422.   I don't bother with HQ because at that time I no longer need any alpha channels.

           

          So, for me, ProRes 422 is a good balance of size and quality.   I don't see any visible degridation in going this route.  I'm sure there are people who have brought in different sequences and used the difference filters and gone pixel peeping, but I'm talking normal viewing visible.   ProRes 422 is still my goto backup/archive format.

           

          This ProRes file will also play nicely with both Compressor and Media Encoder.

          • 2. Re: Exporting Master Archive Copy of Project on Mac
            DMH79 Level 2

            ExactImage wrote:

             

            When I need to store a master copy of a sequence that I know will go to other formats, including Blu-ray, DVD and H264 for the web I export to ProRes 422.   I don't bother with HQ because at that time I no longer need any alpha channels.

             

            So, for me, ProRes 422 is a good balance of size and quality.   I don't see any visible degridation in going this route.  I'm sure there are people who have brought in different sequences and used the difference filters and gone pixel peeping, but I'm talking normal viewing visible.   ProRes 422 is still my goto backup/archive format.

             

            This ProRes file will also play nicely with both Compressor and Media Encoder.

            Okay. Good to know. Will that file play on PC's too (in case I have to give it to a client with a PC)?

             

            Thanks for your input.

             

            Anyone else have input on their methods using a mac?

            • 3. Re: Exporting Master Archive Copy of Project on Mac
              Jim_Simon Level 8

              Any of the three are good options, especially for a Mac user.  If you're concerned about Windows compatibility, then you should skip anything QuickTime, as not every Windows user will have or want to install QuickTime on their machine.  (Plus, it's just an outdated and buggy container format.)

               

              You should also consider MXF OP1a using the AVC-I 100 codec as a viable option.  It's cross platform, Master Quality, and doesn't suffer from any of the issues QuickTime does.  (Plus the files will be smaller to boot).

               

              As for using Comrpessor, you should check out the following:


              http://www.eventdv.net/Articles/News/Feature/Tutorial-Encoding-H.264-Video-in-Adobe-Media- Encoder-CS5.5-and-Apple-Compressor-4-80446.htm

              • 4. Re: Exporting Master Archive Copy of Project on Mac
                Rallymax-forum Level 3

                I agree that OP1a with AVC-Intra is a good plan of those listed. It's the only solution cited that is based on an international standard. That is, not a proprietary compression scheme that may or may not be available in a decade or so.

                 

                The only issue I have with all the suggestions so far is that they all lose chroma detail by sub-sampling to 4:2:2. I can only see 4:4:4 as being an acceptable Archive or DI format.

                 

                If you can devine that you'll only ever need 4:2:2 or 4:2:0 then 4:2:2 is ok - but, if you can't, that's a significant amount of damage done - even if the compression scheme is lossless.

                 

                my 2c.

                 

                Edward

                • 5. Re: Exporting Master Archive Copy of Project on Mac
                  SimonHy Level 2

                  They're all good options. I'm in a mac environment so can vouch that ProRes422 is a good way to go. It was designed specifically as a post production codec (despite the fact that some devices now shoot using it) and has been shown to not have the same generational loss on importing/exporting as other codecs.

                   

                  Rallymax-forum wrote:

                   

                  The only issue I have with all the suggestions so far is that they all lose chroma detail by sub-sampling to 4:2:2. I can only see 4:4:4 as being an acceptable Archive or DI format.

                   

                  I guess it depends on what you use masters for, but if it's graded before turning into a master then it shouldn't be a problem. The extra information in 4:4:4 really only comes into play when you start manipulating the image, every time I've used footage from a master in a new sequence it passes through with no additional effects (because it's mastered already) and thus doesn't have any need of 4:4:4. If you think you'll be re-grading or doing effects and compositing work with the shots then yeah, 4:4:4 is great, otherwise save the disc space.

                  • 6. Re: Exporting Master Archive Copy of Project on Mac
                    DMH79 Level 2

                    Rallymax-forum wrote:

                     

                    I agree that OP1a with AVC-Intra is a good plan of those listed. It's the only solution cited that is based on an international standard. That is, not a proprietary compression scheme that may or may not be available in a decade or so.

                     

                    The only issue I have with all the suggestions so far is that they all lose chroma detail by sub-sampling to 4:2:2. I can only see 4:4:4 as being an acceptable Archive or DI format.

                     

                    If you can devine that you'll only ever need 4:2:2 or 4:2:0 then 4:2:2 is ok - but, if you can't, that's a significant amount of damage done - even if the compression scheme is lossless.

                     

                    my 2c.

                     

                    Edward

                    Interesting. Forgive me but if the video was originally shot on a camera that shoots 4:2:0 color would I still have to worry about it "losing chorma detail by sub-sampling to 4:2:2"? ...or am I mixing things up? Interesting also about the OP1a...never heard of it before and it wasn't brought up much in the other posts so I didn't consider it but I like that you said it a standard that will be around decades from now. And it looks as good/better than the others?

                    • 7. Re: Exporting Master Archive Copy of Project on Mac
                      DMH79 Level 2

                      SimonHy wrote:

                       

                       

                      I guess it depends on what you use masters for, but if it's graded before turning into a master then it shouldn't be a problem. The extra information in 4:4:4 really only comes into play when you start manipulating the image, every time I've used footage from a master in a new sequence it passes through with no additional effects (because it's mastered already) and thus doesn't have any need of 4:4:4. If you think you'll be re-grading or doing effects and compositing work with the shots then yeah, 4:4:4 is great, otherwise save the disc space.

                      For what I'm doing, it will be graded BEFORE making the master. This is just to keep as back up and never to be re-edited again.

                      • 8. Re: Exporting Master Archive Copy of Project on Mac
                        Jim_Simon Level 8

                        they all lose chroma detail by sub-sampling to 4:2:2.

                         

                        That's only true of the UT codec if you use one of it's YUV options.  UT also offers RGB.

                         

                        Plus, with most source material being recorded in YUV 4:2:2 at best, archiving to the same is not a really a major issue.

                        • 9. Re: Exporting Master Archive Copy of Project on Mac
                          Rallymax-forum Level 3

                          Jim Simon wrote:

                           

                          they all lose chroma detail by sub-sampling to 4:2:2.

                           

                          That's only true of the UT codec if you use one of it's YUV options.  UT also offers RGB.

                           

                          Plus, with most source material being recorded in YUV 4:2:2 at best, archiving to the same is not a really a major issue.

                           

                          Yes I agree Jim.

                          I personally have a problem/criticism with Ut RGB since it saves in the RGB colorspace. I just don't think it's a good idea with footage being YUV to be doing a rec601 or rec709 conversion when I know for a fact (as a developer) that the transform matrixes are not always correct - 1 of the 3 years of development of x264pro was getting the colorspace handling correct for every possible pixel format that Premiere/AME can process.

                           

                          I do disagree with the theory that since the source is 4:2:2 or 4:2:0 that you don't need any better. If you do hard cuts and no scaling I'd agree but anything more involves manipulation of the footage as 4:4:4. So, if your long term archive format (ie Master) or short term (ie Diigital Intermediate) is ever going to be manipulated again why damage the result of any of those editing actions (cross dissolves, any Fx, or any Color Correction for that matter)?

                           

                          I fully acknowlege that we're talking about a very small amount of error - and only comes into play if you're compressing the final Master as pretty much lossless but for me I want a Master to be, well, ... a Master. I (personally) define that as being a perfect reproduction of the timeline without the ability to change the editing/Fx decisions made. It's nit picky but it's what I think since storage is DIRT CHEAP. The salary burn in just reading this thread just paid for another 2Tb. We'll all save thousands of RAW camera files (6-10Mb per image) without any concern for size so why are we even flinching at the idea of another Gb here and there for a Master video file?

                          • 10. Re: Exporting Master Archive Copy of Project on Mac
                            Jim_Simon Level 8

                            if your long term archive format (ie Master) or short term (ie Digital Intermediate) is ever going to be manipulated again why damage the result of any of those editing actions?

                             

                            Two reasons.  The first is that any errors are likely to be so insignificant that it won't make any noticeable difference anywhere down the line.  The second is size.  The only way to get 4:4:4 is RGB Uncompressed, and that's a BIG file - about 5x bigger than UT, and about 14x larger than AVC-I 100, with no discernable improvement in quality.  If you're recording RAW, it might make sense.  But for the majority of YUV 4:2:0 source material (or even 4:2:2), it's unnecessary overkill.

                             

                            If anything, the larger concern here would be 8 bit vs. 10 bit.  UT is currently 8 bit only.  That's fine for my 8 bit YUV 4:2:0 source media, but AVCI-100 is 10 bit 4:2:2, and even at 400% pixel peeping is identical to my original.  So it's a pretty good choice as a Master file for many folks.

                            • 11. Re: Exporting Master Archive Copy of Project on Mac
                              Rallymax-forum Level 3

                              FWIW I think we are both in agreement.

                              I was just pointing out that a 4:2:2 output even with uncompressed or lossless compression has loss.

                              For 99% of the time any yuv422 codec will do.

                              Because its a standard not proprietary I prefer AVC over ProRes, DNxHD or Ut.

                              • 12. Re: Exporting Master Archive Copy of Project on Mac
                                ExactImage Level 3

                                Rallymax-forum wrote:

                                 

                                FWIW I think we are both in agreement.

                                I was just pointing out that a 4:2:2 output even with uncompressed or lossless compression has loss.

                                For 99% of the time any yuv422 codec will do.

                                Because its a standard not proprietary I prefer AVC over ProRes, DNxHD or Ut.

                                 

                                ....but will the average Windows client be able to simply play that file or are they going to need more?   AVCIntra requires an entire folder structure, not just a file.

                                • 13. Re: Exporting Master Archive Copy of Project on Mac
                                  Rallymax-forum Level 3

                                  Good morning ExactImage 

                                   

                                  The average Windows client (ie player) will not play any of these suggested Master codecs.

                                   

                                  AVC Intra does not require a folder structure if it is put in the single file alternative - MXF OP1a, or just put it in a MP4 container.

                                  AVC Intra is no different than regular AVC/h.264 except it uses less features of the compression scheme so raw .264 (aka .m4v) or the containers .MP4 or .mts will all work. I don't think MXF is supported by out-of-the-box Windows so mp4 would be the best choice.

                                   

                                  If it's going to a client (ie customer) for review I think the master format is the wrong delivery mechanism for that. If the client wants to see better quality than the stock standard codecs (so AVC 8bit 4:2:0 would be the best choice on Windows) can play then they need to install the Master format's codec to view it in Windows Media Player, QuickTime player or others like VLC.

                                   

                                  My 3c

                                  Edward

                                  • 14. Re: Exporting Master Archive Copy of Project on Mac
                                    Rallymax-forum Level 3

                                    To qualify that.

                                    AVC Intra won't play either on a fresh Windows machine.

                                     

                                    Also I think Jim made a good point re Ut in that it is 8bit.

                                    I think ProRes422 or DNxHD are good fits today since standalone codecs for AVC Intra are not super easy to get. (fresh install windows can't play 10bit or 422 AVC)

                                     

                                    But this only matters if the client wants to view the Master outside of PPro.

                                     

                                    Thus I still prefer AVC Intra 10bit 4:2:2

                                    Ideally I'd use AVC Intra Lossless 10bit 4:4:4 as a Master since it has the best compression scheme in existence today, isn't computationally hard to do , it can be lossless and can also be Intra only for smart transcoding.

                                    CPU speeds are fast enougt that AVC is now a very manageable format especially when constrained to Intra.

                                    • 15. Re: Exporting Master Archive Copy of Project on Mac
                                      Rallymax-forum Level 3

                                      Btw lossless is easier to encode vs 100 or 50 because there is no rate control code. Rate control is the code that decides what parts of the image get the bits and is a significant % of the work.

                                       

                                      I have no transmit concerns so I don't need to be constrained to a set bitrate like 100mbps for AVC Intra 100.

                                       

                                      Cameras have a finite speed they can write to storage thus it us apprpriate for them.

                                       

                                      Once again I'll remind you that I think that this only applies to 1% of the time when quality is more important than any other factor.

                                      • 16. Re: Exporting Master Archive Copy of Project on Mac
                                        Jim_Simon Level 8

                                        AVC Intra won't play either on a fresh Windows machine.

                                         

                                        All you need is VLC, a good idea on any platform.

                                        • 17. Re: Exporting Master Archive Copy of Project on Mac
                                          Jim_Simon Level 8

                                          it can be lossless and can also be Intra only for smart transcoding.

                                           

                                          I don't think that's correct.  Panasonic has developed the new AVC-Ultra standard, which does include a 4:4:4, I-frame only option, but it's not lossless.  I don't think it's possible to use any form of MPEG compression and still be lossless.  To the best of my knowledge, the entire MPEG family of codecs are lossy.

                                          • 18. Re: Exporting Master Archive Copy of Project on Mac
                                            Rallymax-forum Level 3

                                            Jim Simon wrote:

                                             

                                            it can be lossless and can also be Intra only for smart transcoding.

                                             

                                            I don't think that's correct.  Panasonic has developed the new AVC-Ultra standard, which does include a 4:4:4, I-frame only option, but it's not lossless.  I don't think it's possible to use any form of MPEG compression and still be lossless.  To the best of my knowledge, the entire MPEG family of codecs are lossy.

                                            In practice you are correct. If you're talking about the marketing term "AVC Intra 100" then yes it's a definition that is Intra-frame only + 10bit + 4:2:2 at a Constant Bitrate of 100Mbps.

                                             

                                            The combination of AVC + Intra frames only + 10 or 12 bit + 4:4:4 + X constant bitrate is being marketed as "AVC Ultra".

                                             

                                            Outside of these marketing labels there is no reason you can't mix any of AVC's capabilities.

                                             

                                            re MPEG always being lossy - that's not true. You can do variable bitrate encoding with the rate factor to zero (thus resulting in 0 dB error ie mathematically lossless) to get "Lossless".

                                             

                                            Panasonic and others see the compression noise/errors/artifacts created at 100Mbps as being acceptable for a camera thus we have "AVC Intra 100".

                                            "AVC Ultra" (which I seem to recall is 250Mbps) takes it to the next level with higher resolution (4K I think), higher bitrate (10 or 12bit I think), and higher chroma detail (4:2:2 or 4:4:4).

                                            • 19. Re: Exporting Master Archive Copy of Project on Mac
                                              DMH79 Level 2

                                              Rallymax-forum wrote:

                                               

                                              To qualify that.

                                              AVC Intra won't play either on a fresh Windows machine.

                                               

                                              Also I think Jim made a good point re Ut in that it is 8bit.

                                              I think ProRes422 or DNxHD are good fits today since standalone codecs for AVC Intra are not super easy to get. (fresh install windows can't play 10bit or 422 AVC)

                                               

                                              But this only matters if the client wants to view the Master outside of PPro.

                                               

                                              Thus I still prefer AVC Intra 10bit 4:2:2

                                               

                                               

                                              So bringing it all back home...

                                               

                                              Assuming the clients don't have PPro or any other editing software, just regular people who know nothing about codecs, etc. that want to have a copy of the video that will last for years and can maybe one day be converted into some other format if necessary...but will work on both mac and PC computers...

                                               

                                              ProRes 422 or DNxHD are good ways to go.

                                               

                                              MXF OP1a using the AVC-I 100 codec may actually be the best option but clients that don't have PPro may run into issues trying to view it. Unless they get a VLC player of course.

                                               

                                              Am I right given my purpose (archive copy for non tech-savy clients to view on PC/Mac easily AND maybe one day convert)?

                                              • 20. Re: Exporting Master Archive Copy of Project on Mac
                                                ExactImage Level 3

                                                Hah ha  

                                                 

                                                I think you came around to my way of thinking after all

                                                • 21. Re: Exporting Master Archive Copy of Project on Mac
                                                  Jim_Simon Level 8

                                                  ProRes 422 or DNxHD are good ways to go.

                                                   

                                                  Not really.  They both require the installation of the codec, and QuickTime.  That may be a 'harder sell' than a simple player like VLC.

                                                  • 22. Re: Exporting Master Archive Copy of Project on Mac
                                                    ExactImage Level 3

                                                    ProRes merely needs Quicktime, which is a player, just like VLC, and a suprising number of people already have it.

                                                     

                                                    DNxHD does require codecs on to top Quicktime which is why I would prefer ProRes over DNxHD.   However, there is no one absolute best here (as people are starting to realise).   For client reviews, H.264 / MP4 works about as good as anything.  For archival and 'possible' client viewing then ProRes works well for me and it's easy to generate an H.264 file from relatively quickly without needing to load it back in to Pr.

                                                     

                                                    Other file types have advantages and disadvantages too.   One man's preferred codec is another man's reason to bury his head in his hands and weep.  Do what works for you.  Nothing more, nothing less. 

                                                    • 23. Re: Exporting Master Archive Copy of Project on Mac
                                                      DMH79 Level 2

                                                      So Jim,

                                                       

                                                      MXF OP1a using AVC Intra 100 is your choice.

                                                       

                                                      ProRes 422 is ExactImage's choice.

                                                       

                                                      Each work well, look good, and are valid options for me to choose from. Got it!

                                                       

                                                      Thank you!

                                                      • 24. Re: Exporting Master Archive Copy of Project on Mac
                                                        Jim_Simon Level 8

                                                        ProRes merely needs Quicktime, which is a player, just like VLC

                                                         

                                                        QuickTime is more than just a media player like VLC, it a full multimedia framework.  And a buggy one at that.

                                                        • 25. Re: Exporting Master Archive Copy of Project on Mac
                                                          josephs51576386 Level 3

                                                          I personally use DNxHD as my master format for actual "real" footage most of the time. However if you're wanting to store graphics I find that Lagarith and UT are much better at this task. You end up with a file that is MUCH smaller in most cases and is still lossless. However I wouldn't ever recommend giving a client DNxHD to view, because they would have to install DNxHD on their system. Then the issue with ProRes for client viewing is that they would have to have Quicktime installed on their system if they owned a PC. (Which in the past I have had quite a few clients who didn't have Quicktime) I also wouldn't recommend giving a client Lagarith or UT either though since they would have to install  a codec to view the media.

                                                           

                                                          So for my clients I always want them to be able to view their media easily therefore I always provide them a h.264 copy inside of a .mp4 container, simply because it will play in most scenarios regardless of their system setup. So really in the end unless your client is tech savy I wouldn't ever give them a master copy of the file for viewing purposes unless they request it. I would however provide them a high quality master, I just wouldn't provide it for their viewing copy of the end product. I used to do this and I quickly realized it was very stupid to do so, I always ended up having to explain things that I could of avoided having to explain. I also ended up spending extra time trying to explain what they needed to do etc... I always give them their stuff, then explain what each copy is. I always place each file in its own folder as well. "For viewing" "High Quality Master" "web copy". The other great thing about this though, is that you can charge for this service. I also offer them a web copy of the media as well. Which is also a way to make a little extra money in some cases.