15 Replies Latest reply on Oct 28, 2015 4:10 PM by KS555

    Proxy codec for AVCHD editing on iMac?

    codexplorer Level 1

      Hi there,

       

      as I have discovered (and posted on other threads), my brand new iMac 21.5" is unfortunately not capable of smoothly editing AVCHD footage natively (hard drive config, no hardware Mercury Engine Acceleration, etc). So I switched to transcoding into a QuickTime format...

      I am coming from FCE so I have only Apple Intermediate Codec available on my computer (plus those coming with CS5.5). I also downloaded Avid's codecs and Cineform's. The latter won't work for me, so I may post questions about this in a different post.

      Anyhow, transcoding works fine and I can edit with smooth playback in PP BUT, of course, I get massive files. I can't seem to find any "proxy" codecs in the list offered in AME, so I am trying to figure what kind of optimization I can do to get smaller transcoded files while still having something watchable (as a preview in PP) and giving me enough quality to edit (currently I am mostly using color correction and stabilization, on top of the usual titling, transitions, etc). Note that I replace the transcoded files by the original footage before exporting in AME to preserve the original quality.

       

      Below, I will quote the  expansion ratio I obtain for a 130 MB 1080i60 AVCHD clip (Sony HX5V, 17 Mbps) using a 50% quality setting in AME (although the quality settings doesn't seem to affect the final size much if at all):

      Apple's AIC: 5.2 (~120 Mbps)

      Avid's DXnHD's 145: 7.6 (~147 Mbps)

       

      Note that I listed this results in decreasing order of processing speed. The visual quality appears similar to my untrained eyes (on the test clip I used, at least). I haven't tried other codecs (the list is quite long and not all of them are good for 1080i footage).

      I noticed that FCP comes with a ProRes "proxy" codec with a bit rate of 45 Mbps, which would give an expansion factor of 2.6, which is a significantly smaller expansion factor (twice smaller than AIC).

       

      Now, of course I could try transcoding in a non-QuickTime format. MPG2 comes to the mind. Its default bitrate is in the 15 Mbps! But I haven't tried it yet in a PP timeline, so I am not sure it is any easier to edit with my iMac than the original H.264 (AVXHD) clip.

       

      - Is it worth for me to spend $50 on Compressor to get access to this codec?

      - Is there a free codec (or a combination of settings for some of the codecs coming with CS5.5) which would result in a similar expansion ratio?

      - Am I asking the wrong question or missing some important points?

       

      Thanks for any feedback.

        • 1. Re: Proxy codec for AVCHD editing on iMac?
          Colin Brougham Level 6

          You can certainly use DNxHD or ProRes (if you want to pony up for it); they're pretty similar in most respects. The biggest issue is that Pr doesn't have native 64-bit decoders for those codecs, so you'll be relying on QuickTime to decode them; this means you're going through the 32-bit bridge application that Adobe wrote. It's really not that big of a deal: I'm working on a longish documentary project with ProRes captures (SD, however) on a PC, and performance is amazingly good. So, it's certainly worth a try with either of those codecs.

           

          You could certainly give MPEG2 a shot, especially at a bitrate of 100Mbps or higher (though I think Pr tops out at 100Mbps, anyway), using at least the High profile and High Level (going to the 4:2:2 profile would let you increase the bitrate up to 300Mbps) and by creating an I-frame only file. To create an I-frame only MPEG2 stream, set the M frames and N frames values to both 1. That way, each frame is independently compressed from any other frame. Note that your audio options are Dolby Digital (AC3) and MPEG-1. This will create an MPG file that imports into Pr without a problem, and it will also be decoded with the 64-bit MPEG2 decoder.

           

          The files will get larger--at 100Mbps, your filesize will be about 4 times larger than the original AVCHD clip, for example--but MPEG2 is easier to decode than H.264. You'll have to do some experimentation, but it might provide a little better editing experience than what you're having now.

           

          Hope that helps somewhat...

          • 2. Re: Proxy codec for AVCHD editing on iMac?
            codexplorer Level 1

            Colin, it definitely helps, as I have neither read anything about MPEG2 parameters yet nor tried it in a timeline. But 100 Mbps is still close enough to AIC. I tried to set the target bitrate to 50 Mbps (and N=M=1) in AME and surprisingly:

            1) it works

            2) it even plays within the Finder... Not sure why, as my other mpg files will usually try to play via MPEG Streamclip and the latter shouts that it needs the MPEG2 codec (!?).

            Anyhow, you are telling me that this kind of file should play like butter in Premiere? That's good news, but of course I need to verify that!

            Thanks.

            • 3. Re: Proxy codec for AVCHD editing on iMac?
              codexplorer Level 1

              Follow up question: now that I am able to edit using a proxy format that does not clutter my hard drive, I am running into an unforeseen issue related to dynamic link.

              I have fallen in love with the Wrap Stabilizer of AE, which I am also using for some color correction. Therefore, I am heavily using "Replace with AE Composition" in Pr.

              BUT, as I am using proxy footage, which I later replace with the original ones before exporting the edited sequence, I am facing the following issue (see the end of the discussion here):

              - suppose I want to create a sequence using footage "clip.mov".

              - to allow smooth editing, I create a proxy "clip.mpg" that I put into my sequence

              - since I want to stabilize (for instance) that footage, I replace it in the sequence by a linked AE comp.

              - when I am satisfied with the whole sequence, I "Replace footage..." all clips in my Pr project.

              - I export the sequence to media...but the clips that have been replaced by AE comp will now be rendered WITH THE PROXY FOOTAGE!

              In other words, replacing the proxy by the original footage within Pr has no effect on the AE comp (which I am in fact using only in Pr).

              From the answer that I got in the discussion linked above, it appears that this is the expected behavior. However, I would question the logic of this, considering the intended purpose of dynamic link (which is to avoid having to render something in AE, then reimport it into Pr and have to do that over and over again each time some modification needs to be done in the AE composition).

              Am I missing something? Is this worth a feature suggestion, as implied in the above discussion?

              • 4. Re: Proxy codec for AVCHD editing on iMac?
                DerikSavage Level 1

                Just getting into offline editing, and yeah... this is terrible. 

                 

                I've been able to find a workaround for it, but it's not a very user-friendly fix.  I replace the footage in AE, and rescale the nested comp.  Then RAM Preview so Premiere recognizes the change.  This is fine if you're just working with a few linked comps, but any more than that and it becomes a huge and tedious pain.

                 

                Would be nice to have Premiere/AE handle all of this under the hood. 

                • 5. Re: Proxy codec for AVCHD editing on iMac?
                  codexplorer Level 1

                  Would you mind expanding a little? What do you mean by "rescaling" the comp? In Pr?

                  My problem was that replacing the clips in AE and trying to get Pr to react to that led to it basically freezing (can't remember the symptoms, but the bottomline is that I gave up).

                  To avoid the problems I was describing, I changed my workflow (after some major hardware upgrade): I transcode in ProRes (got the codec from the FCPX trial) and just stick to it (no replacement at the end). Of course the file size explodes, but that's fine, as I just put those on a new 4T Thunderbolt HD which sustains the data transfer rates perfectly. Now I only need a second monitor for preview and I am set.

                  But this still shows that a mere iMac is NOT a good platform for editing... I just wished Adobe published a benchmark of the Mac version their software suite on current Apple hardware (for future users).

                  • 6. Re: Proxy codec for AVCHD editing on iMac?
                    DerikSavage Level 1

                    I feel your pain.  I spent a crapton of money on upgrading my computer to try to edit AVCHD files natively (multicam no less), and it still wasn't enough.  That's when I realized I was doing it wrong and that a good offline editing workflow can literally save me thousands and cut my turnaround time considerably.  Even though you've got the hot new badness happening with your thunderbolt drive and your ProRes through all dem CUDA cores, one day even that will be on the verge of obsolescence and you'll be spending more on new hardware to try to keep up.  But the truth is, you can edit just fine on an iMac.  Hell, you can edit just fine on an eMac if you're willing to scale your footage down enough.  The key is to only process what you have to and not throw huge files at your comp if you can avoid it.  But I digress....  Here's my workflow based on your scenario above. 

                     

                    Let's say for instance, I've got original footage at 1920x1080 and my proxy files are 25% of that, or 480x270.  I'm using Cineform (I'm on a PC), so everything is .avi. 

                     

                    The sequence settings I'll use in Premiere are based on the original footage, not the proxy.  So when I create my sequence, I'm creating a sequence that is 1920x1080 and dropping my 480x270 proxy files into it.  This will cause the footage to be very tiny in the program monitor, so I'll want to right-click the clip(s) and "Scale to Frame Size".  This causes the footage to scale up to whatever the sequence is set to.  Now you've got buttery-smooth playback and your computer won't break a sweat.

                     

                    Now let's say you've got a shaky shot you want to stabilize in AE, so you "Replace with After Effects Composition".  This is where things get a little more complicated. 

                     

                    When Premiere sends the clip over to AE, it sends whatever you're working with in Premiere. In this case, the proxy file.  And if you replace that file later in Premiere, After Effects doesn't care.  It keeps whatever you originally sent it.

                     

                    Okay, fine.  We can deal.

                    So After Effects creates a new comp and imports your footage into the AE project panel.  And what you're looking at now in the timeline is "Project Name Linked Comp 01". 

                     

                    You'll also notice in the AE Project Panel that you have three things:  your proxy clip, a comp with the same name as your proxy clip, and your linked comp. (It gets more complicated when you send over multicam clips, but the idea is the same).

                     

                    So what's happening here is that the clip inside your linked comp is actually another comp that contains your proxy file. 

                     

                    Now to replace that proxy file with the original full res version.  Just go the AE Project Panel and right-click the file, then choose "replace footage > file".  Navigate to the full res version and select it.  Bam.  Done.  No. Wait.  Now the footage is blown up to 400% and all you're seeing is the center of your image.  That's because you "Scaled to Frame Size" in Premiere before sending it over.

                     

                    You see, the comp with the same name as the clip holds whatever attributes the clip had in Premiere, including the scaling.  So go to that comp.  The nested one inside your "Project Name Linked Comp 01" comp.  Select the one and only layer in the comp.  Hit "S" because you know your shortcuts.  Now change the scale to 100%. 

                     

                    Don't forget to RAM Preview your whole comp before going back to Premiere. Otherwise it might not recognize the changes you've made.

                     

                    BTW, you don't have to do all of this until you're ready to render out your project.  If you want to do all of your AE work on your proxy file and swap it out later, go for it.  Just make sure you're applying your effects and whatnot to the comp, and not the footage itself.

                     

                    Easy enough, right?  Now go back into Premiere and do your thing. 

                    • 7. Re: Proxy codec for AVCHD editing on iMac?
                      Fuzzy Barsik Level 4

                      Guess, by decreasing the frame size you outmaneuvered yourself a bit.

                      What you actually achieve with that is decreasing bitrate, since lower bitrate is required for encoding smaller frame size.

                       

                      Let's have a look at the task from another angle.
                      PrPro creates MPEG2 media files for a preview, which have overal bitrate of about 25.5 Mbps and the following GOP: M=1, N=3.

                       

                      Since we need a proxy, we can easily compromise on some quality. Therefore, lets transcode our footages to Intra-frame MPEG2 (i.e. M=1, N=1), but with around 17 Mbps bitrate instead of 100 Mbps or higher as Colin was talking about. And what we have now? We have media files with original frame size, similar to original footages or even smaller file size and smooth timeline playback - with high probability the Timeline will have grey not even yellow bar.

                      • 8. Re: Proxy codec for AVCHD editing on iMac?
                        DerikSavage Level 1

                        I tried your suggestion with using MPEG2 proxies in a test sequence, and it worked great!  I was able to play back 16 angles in multicam without any problems.  It was just as good performance-wise as what I had done with my Cineform proxies.  The difference is that the MPEG2 proxies looked much better because they weren't low-resolution like the Cineform proxies.  Files are larger, but whatever.  They'll get deleted when the project's complete anyway.

                         

                        The problem now seems to be the fact that Premiere won't easily relink to different file extensions. My original footage was AVCHD and has an .mts extension.  Also have a few .mov and .mp4 files thrown in.  My Cineform HD files are .avi, and the proxies I just made are .mpg.  If I were to offline my proxies and try to relink them to the .mts files, it would ask me for the location of every single file.  I've got well over a hundred files for my current project, and this does not sound like a good time.

                         

                        The advantage with using Cineform is that it's all .avi.  Both the proxies and the full res files.  Relinking to the full res versions is no problem.  But if I start with an mpg and try to link to an avi, it makes me go one at a time.  No fun at all. 

                         

                        I'm sure there's a good solution to all of this, but it eludes me.  How does one edit with full res proxies while still being able to relink to the original media or a good intermediate with little to no hassle?  I feel like I'm close on this one, but any help/advice would be great.  I know very little about optimal bitrates/GOPs for proxies, so that might also be good information.

                         

                        Good to see this thread is developing into a useful discussion.

                        • 9. Re: Proxy codec for AVCHD editing on iMac?
                          Fuzzy Barsik Level 4

                          1. Bear in mind that you control the file size via bitrate. If your CineForm proxies have smaller file size than MPEG2 ones, they have lower bitrate. Experiment with the bitrate settings so as to find optimal ratio 'acceptable quality/file size'.

                           

                          2. No, PrPro doesn't offer proxy workflow, and in my humble opinion Colin properly outlined it here why.

                          The workaround is the following:

                          - create a folder for your proxies next to the folder containing original footages and name it e.g. AVCHD.Proxies;

                          - transcode your MTS files into MPEG2 ones with the same file names and MTS file extention (or run Batch Rename in Bridge after transcoding with MPG file extention);

                          - close your PrPro project, created out of original MTS files;

                          - temporarily rename AVCHD folder containing original MTS files to e.g. AVCHD.Sources and AVCHD.Proxies to AVCHD;

                          - open up PrPro project and enjoy: all your original AVCHD files are now linked to MPEG2 proxies.

                           

                          When you're done, reverse rename folders.

                          • 10. Re: Proxy codec for AVCHD editing on iMac?
                            codexplorer Level 1

                            Well, I am not too excited by the workflow suggested by MEM, to say the least... But I fully admit that I am looking at the increasing bitrate of cameras in my price range with dread... and exhilaration at the same time!

                            For the time being, I will stick to transcoding to ProRes as it suits my needs. I don't think color grading or checking for out-of-focus footage (I am doing alot of underwater videos where it's hard to gauge whether focus is perfect while shooting) on a low res clip will do me any good.

                            Now, I know that CS6 as incorporated warp stabilization in PP, so that would save me one step. But since I do some color grading in AE, not so much.

                            • 11. Re: Proxy codec for AVCHD editing on iMac?
                              Fuzzy Barsik Level 4

                              Proxies are not for grading. Rather for rough cuts and motion check.

                               

                              By the way, where are you diving?

                              • 12. Re: Proxy codec for AVCHD editing on iMac?
                                Nick Papps Level 1

                                I transcoded my footage to ProRes 422 to try for this project and it seemed to change the size to somthing other than 1920x1080 and it appears like it changed my sequence settings.  Can I fix the settings again when I reconnect to the original footage for grading-rendering, or am I stuck?

                                 

                                Also, if i dynamically link to do some After Effects work on a few clips should I relink to the original footage first and then go to AF to get the better quality or will it link to the higher quality footage later when i reconnect?

                                • 13. Re: Proxy codec for AVCHD editing on iMac?
                                  Fuzzy Barsik Level 4

                                  1. Transcoding per se can't change frame size. If the frame size was changed, it were you who set it that way (either intentionally or not) in exporting settings dialog.

                                   

                                  2. Sequence settings in PrPro can't be changed after the sequence was created. If a sequence was created with 1920 x 1080 frame size settings, it remains 1920 x 1080 irrespective of the footage frame size you drop into it.

                                   

                                  3. For dynamically linked comps replace original footage with a proxy and back right inside AE project. Unlike PrPro, AE offers dedicated proxies and placeholders workflow.

                                  • 14. Re: Proxy codec for AVCHD editing on iMac?
                                    Nick Papps Level 1

                                    What's the best way to relink all the clips in my project to then send to Speedgrade?  The tech help people say i have to do it with each individual clip in my bins and then "replace from bin" in my timeline after that.

                                     

                                    This sounds long and painful.  Is there an easier way? Unlink relink maybe?

                                    • 15. Re: Proxy codec for AVCHD editing on iMac?
                                      KS555 Level 1

                                      Did anyone in this discussion ever find a solution?  What were your settings?

                                      I'm trying to do a proxy edit on AVCHD files, but am having problem with it recognizing stitched files.   Posted on the issue here  CreativeCOW    but received no responses.

                                      It looks like here  After Effects Help | Working with footage items   there is a workflow for CC, but I am in CS6 and that is what I have to work with for now.

                                      Anyone out there have a solution for stitched proxy mts files?  Thank you