7 Replies Latest reply on Dec 11, 2009 6:23 AM by the_wine_snob

    Why is export relatively slow even when not recompressing uncompressed source?

    dmsketchup

      I have captured Hi8 video through an ADS Pyro AVLink directly into PE4.  (By the way, this works brilliantly and thanks to those in this forum who have pointed the way to this solution).

       

      The resulting .avi files (DV Type 2 AVI) on the computer hard disk are huge (gigabytes) as expected, so I am, perhaps naively, assuming they are more or less uncompressed.  PE4 absolutely flies when using this type of source clip (uncompressed AVI ??), even with 2 hours of video on the timeline.  (When I have used other .mpg files as source clips, things bog down with only 20 minutes on timeline). 

       

      So, however I export as discussed below, I would like to end up with .avi files just like the raw captured ones, so that in my final big project, PE4 doesn't bog down.

       

      OK, the raw captures appear on the PE4 timeline, and I am not adding any effects or transitions.  I just want to export sections of the timeline (using the work area bar) to separate .avi files on the computer hard disk.  Later, in a new project, I will reassemble all these exported .avi files.

       

      It seems that no matter which export file type I select (File-Export-Movie-Settings-General-File type), even if I select Microsoft Uncompressed AVI and uncheck "Recompress", the export procedure takes a long time with the progress bar saying it is "rendering frame x of so many...".  It may be a basic concept I am missing, but why does exporting like this take so long?  I could understand it if I was telling it to export as .wmv or .mpg or something, but I just want to export raw uncompressed avi from raw uncompressed avi.  It seems this should only take as long as a simple DOS file copy command should take.  No?

       

      The export procedure is currently a real bottleneck for me.  Is there a solution from withing PE4? Or perhaps I should be pre-processing (cutting) in another application?

       

      Thanks for any help,

      Dave

        • 1. Re: Why is export relatively slow even when not recompressing uncompressed source?
          Steve Grisetti Adobe Community Professional

          The AVIs you're capturing with the Pyro are not uncompressed, despite being so large. They are compressed with the DV codec (DV-AVIs), which are about 1/5 the size of uncompressed AVIs.

           

          As you've noted, these are indeed the ideal video files to work with in this program -- as indicated by the fact that there is no red or green line above the clips on your timeline unltil you add effects or transitions.

           


          So the format you want to output, if you want to create similarly formatted video for another project, is DV-AVI, not uncompressed. And that should output relatively quickly from those source files.

           

          You don't say which version of the program you're using, but how you ouptut DV-AVIs varies from version to version. Prior to version 4, you selected the Timeline and went to File/Export/Movie. In newer versions you go to Share/Personal Computer/DV-AVI.

          • 2. Re: Why is export relatively slow even when not recompressing uncompressed source?
            dmsketchup Level 1

            Thanks Steve. The fact that they are actually compressed answers alot of questions.

             

            I have Premier Elementes 4.  I can either Share-Personal Computer-DV AVI or File-Export-Movie.  The computer is a 2.66 GHz dual core.  I have a separate drive for media, and it has lots of unfragmented room.

             

            I justed tested both methods exporting a 2 minute section of the timeline.

             

            The Share-Personal Computer-DV AVI took exactly 2 minutes (!!??).

             

            The File-Export-Movie, choosing Microsoft DV-AVI, no recompress, and no still optimization took about 1 minute 55 seconds (!!??).

             

            So both methods are essentially as slow as the original capture (real time from Hi8).

             

            Does that sound right?

             

            Also, when one goes the Share-Personal Computer-DV AVI route, there doesn't seem to be any settings to tweak.  Whereas going the File-Export-Movie route, there are numerous settings, including the option to not recompress, which everyone seems to recommend.

             

            So where does Share-Personal Computer-DV AVI get its settings from?  And in particular, does going the Share-Personal Computer-DV AVI risk violating the 'thou shalt not recompress when exporting' rule?

             

            Thanks again for the excellent support.  This forum is terrific.

             

            Dave

            • 3. Re: Why is export relatively slow even when not recompressing uncompressed source?
              Steve Grisetti Adobe Community Professional

              I'm surprised that a fully rendered timeline needs that long to export/share. Though a 2 minute segment may not be a good test. Try a 15 minute segment. I would expect that it won't take much longer to output that your 2 minute segment.

               

              Share is a simplified version of export. In fact, I wouldn't dig too deep in Export/Movie's settings. You're best just using one of the basic presets in Share.

               

              Glad you like the forum. Please also check out the site I co-founded to support my books, Muvipix.com. It has one of the most active, helpful, knowledgeable forums you'll ever see! I highly recommend you drop by and say hello.

               

              In addition to the forum, you'll find lots of helpful products including tutorials, tips, custom-created DVD templates, licensed music, sound effects and stock footage. (And, of course, there are my books.) Much of it is free, and even more is available for a low annual subscription.

               

              But at the very least, drop by and say hi. If you really want to learn a lot about this program (or just meet some great people) it's the best community on the web!

              • 4. Re: Why is export relatively slow even when not recompressing uncompressed source?
                dmsketchup Level 1

                Well, the 15 minutes on the timeline is taking darn close to 15 minutes to export.

                 

                Oh well, if you have any other ideas, let me know.  Otherwise, I'll just deal with it.  There was lots of good information in this thread, so I'll mark it as answered.

                 

                Thanks again, and looking forward to checking out Muvipix.

                 

                Dave

                • 5. Re: Why is export relatively slow even when not recompressing uncompressed source?
                  the_wine_snob Level 9

                  Dave,

                   

                  The two determining factors with Export/Share, will be your CPU and your I/O sub-system, i.e. your HDD's, their speed, controller type and how allocated.

                   

                  You indicate that you have two physical (not partitions on one physical HDD) HDD's. How is your system set up, i.e. your Page File, location of OS, programs, Project, Scratch Disks and your Export files? There could well be ways to speed things up.

                   

                  In another thread on multiple HDD's, I made mention of some ways to speed the process up to its max.: my minimum would be 3x SATA II HDD's, split this way:

                   

                  C:\ OS, programs, and probably the Page File

                  D:\ Project & Scratch Disks*

                  E:\ Media

                   

                  Now, with more HDD's, you could do something like this:

                   

                  C:\ OS and programs

                  D:\ Page File

                  E:\ Project & Scratch Disks

                  F:\ Media

                  G:\ Export files

                   

                  In a perfect world, D:\, F:\ & G:\ would be RAID 0 (different 0 types, depending on the RAID controller, with probably a RAID 50 being best).

                   

                  * Some additional speed increase could possibly be realized, if one had a separate HDD for the Scratch Disks, but then deleting older Projects becomes a bit more involved. While I do not have benchmarks on the location of Scratch Disks, I doubt that their placement elsewhere would be something that one could tell, just with observations.

                   

                  Good luck,

                   

                  Hunt

                   

                  BTW - Real Time is about the norm, on most systems. My workstation with 6x internals, with full splits, runs about 75% of RT. An improvement, but not Earth-shattering. One would probably get that sort of a result with a Quad-core, or even better, dual i7's. I'd guess that with the proper I/O and dual i7's, one could get down to about 50% of RT.

                  • 6. Re: Why is export relatively slow even when not recompressing uncompressed source?
                    dmsketchup Level 1

                    Thanks Hunt.  I have two physical drives.  I bought the second one after seeking advice on this forum a few months ago.  Even having just a second drive increased PE4 performance significantly - another great result from this forum.  I followed whatever advice there was at the time (can't remember exactly), but basically I have the program on one drive, while the other drive is reserved exclusively for media (and whatever scratch/temp files would have been recommended at the time).  I did not spring for more drives as I planned to replace the computer soon.

                     

                    Anyway, regarding export speed, and looking at my 100% real time vs your 75% realtime vs a best case of about 50% real time, I guess my question is really answered: - exporting takes a long time regardless of source, destination, and hardware.

                     

                    So, now I'd like to ask if there is a reputable third party (preferably inexpensive) program that could just lift the clip data directly out of the original .avi file and store it in another .avi file (of the same format, compression, and whatever).  I would think this could be done at well less than 1% of realtime.  The new .avi files would then be imported into PE4 just like the original was, final editing applied, and then finally exported/burned to dvd (in realtime).

                     

                    So I would capture a 2 hour tape through the ADS PyroLink giving me a multi-gigabyte .avi file.  I would read this .avi file into the third party software, quickly navigate to the 10 minute segment I wanted, set start/stop points, lift the section out and and save the clip in another .avi file (in seconds, not minutes).  The key point is that I could then delete forever the multi-gigabyte original file to free up harddisk space.  I would repeat this procedure with other big .avi capture files.  Finally, I would take all my smaller clip .avi's, assemble them in PE4, final edit, then export/burn (in about realtime).

                     

                    Basically, I have about 50 Hi8 tapes to get through.  Each one is about 2 hours long and might have only 3 clips of interest (each about 10 minutes long lets say), all on different subjects (subject1, subject2, subject3) (destined for 3 different dvd's for example).   After 50 Hi8 tapes, I might end up with 8 clips on subject1.  If I go the normal export route in PE4, that's 80 extra minutes spent exporting clips during the processing of the 50 tapes (just for subject1).  However, if I go with the magic preprocess software, its just a few extra seconds.

                     

                    I'm trying to avoid having to keep track of 50 x 20Gb of files on my drive at once, and I also think having a PE4 project with 8 20Gb source files in it might (for the sake of 80 minutes of final video) would be odd.

                     

                    Hope this makes sense, and thank for the extra info.

                     

                    Regards,

                    Dave

                    • 7. Re: Why is export relatively slow even when not recompressing uncompressed source?
                      the_wine_snob Level 9
                      I have the program on one drive, while the other drive is reserved exclusively for media (and whatever scratch/temp files would have been recommended at the time).  I did not spring for more drives as I planned to replace the computer soon.

                       

                      There is a slight bottleneck, as the computer must access both the Project file (plus Scratch Disks) and the media files. That is where splitting the duties even more comes in. However, as you report, you are looking to upgrade the computer soon. I understand that, as I will be doing similar in the near future. One consideration is that SATA II (if that's the controller that you have), is about as good as it gets right now, and any SATA II HDD's, that you buy can be incorporated into the new computer with no loss in anything. Just a thought.

                       

                      now I'd like to ask if there is a reputable third party (preferably inexpensive) program that could just lift the clip data directly out of the original .avi file and store it in another .avi file (of the same format, compression, and whatever).  I would think this could be done at well less than 1% of realtime.  The new .avi files would then be imported into PE4 just like the original was, final editing applied, and then finally exported/burned to dvd (in realtime).

                       

                      As for extracting the data, I'm not sure what might be out there. There are other editing/utility programs, like AVSynth, and Womble, but I am not sure if they would do what you want. I just have zero experience with them. My workflow is pretty set (old dog vs new tricks sorta' thing), so I do not experiment with other programs, unless I feel a compelling need to do so. Maybe others could help. I do not think that any would be THAT much quicker, but could be very wrong.

                       

                      So I would capture a 2 hour tape through the ADS PyroLink giving me a multi-gigabyte .avi file.  I would read this .avi file into the third party software, quickly navigate to the 10 minute segment I wanted, set start/stop points, lift the section out and and save the clip in another .avi file (in seconds, not minutes).  The key point is that I could then delete forever the multi-gigabyte original file to free up harddisk space.  I would repeat this procedure with other big .avi capture files.  Finally, I would take all my smaller clip .avi's, assemble them in PE4, final edit, then export/burn (in about realtime).

                       

                      Except for the lightening-fast Export aspect, this is basically what I do with clients' VHS tapes. I capture to DV-AVI Type II's, and archive these on my externals. I do a capture log for each one, noting the TimeCode and details of the elements that I will want. I then go though these capture files (in PrPro for me), and set my In & Out Points for multiple "instances" of the main Clip, resulting in several Trimmed mini-Clips. These are then either Exported, or in PrPro, Imported into a new Project (nice feature of the big-brother program), and edit.

                       

                      In PrE, you can do almost the same thing (except for the Importation of a Project's Sequence into a new Project), resulting in say 5 mini-Clips on the Timeline, each being a different instance of the "master" Clip. These would then be Exported to DV-AVI Type II files for use in a new combined Project. This does NOT get you around the Real Time, or close, Export, but that is what coffee is for - right? When I'm doing this, I just jump on the laptop and come here, while my workstation is Exporting away!

                       

                      Not what you're looking for, and I'll be interested in seeing what might be available out there. One thing that would get you closer to your ideal would be PrPro CS4.2. As of CS4, all Exporting was added to Adobe Media Encoder, which is now a stand-alone app. One can load up its queue with many Sequences (think mini-Projects in PrE), and just start it up Exporting. While it's doing its thing, you can still edit in PrPro, provided that your machine is stout enough. That is about as close to that ideal, as I can come up with.

                       

                      Good luck, and I'll be watching to see what ATR, Steve and others have to offer - might be some neat freeware program just a URL away?

                       

                      Hunt