6 Replies Latest reply on Feb 22, 2012 3:02 AM by mjjwatson22

    AE CS5 rendering speeds with 100+ file sources, no effects

    mjjwatson2

      I'm trying to render a 1080p, 5 minute project that consists of 144 1080p videos, all masked so only a part reveals of each.

       

      That is it, there is no effects, nothing to process, just that huge amount of source videos.

       

      My computer is a 2010 macbook pro with 8gb of ram and can certainly handle 'normal' projects, but this seems to be impossible to render. About 250 hours, is the estimate.

       

      I'm assuming this isn't a ram issue, it's the trouble of accessing the files on the hard disc. Do you have any suggestions of how this could be done? Would an SSD significantly improve things? Or having the files on a partition?

       

      Any thoughts would be greatly appreciated.

        • 1. Re: AE CS5 rendering speeds with 100+ file sources, no effects
          Mylenium Most Valuable Participant

          nothing to process

           

          Erm, as per your own words: masks?! Transparency blending calculations don't fall out of the sky, you know. Try to imagine for one second how long it might take to rasterize a vector based mask including antialiasing and blend that with the result of the underlying layers also having gone through these steps. Not saying that it should make your computer explode, but you couldn't be more wrong about "nothing to process". And even decoding compressed, chroma undersampled footage and reconstructing the missing color information is "processing" of sorts just as is going through these steps for output compression. So, like it or not, it will take some time on that end as well. I tend to agree, though, that your footage I/O is probably killing everything. I don't see an easy solution on that one. It's a complex thing:

           

          • AE needs to read the footage
          • it needs to decode your compressed format
          • decoding compressed formats many times requires to decode extra frames, not just the ones actually visible/ used
          • decoding such stuff in AE invokes MediaCore
          • MediaCore writes conform and temporary files to the MediaCache to accelerate file access and may also fully decode any audio info
          • on top of it you are trying to write your output files to an already heavily burdened drive

           

          All of this means that a simple SSD won't do you any good. There is simply too much overall activity. Despite what marketing myths may suggest, your only reasonable option is to buy an external RAID system, preferrably one based on four disks rather than just two. Then you could move all your sources to this drive and also assign it as the location for the MediaCache, only using your internal drive as the output location for the final file. In any case, have a look at your activity monitor before taking any action and also experiment with things. E.g. outputting to a different format may already improve matters. Use an image sequence in favor of clip based formats. and just in case we missed the obvious: Turn off MP rendering. Quadrupling the pain by using multipel render processes surely doesn't help...

           

          Mylenium

          • 2. Re: AE CS5 rendering speeds with 100+ file sources, no effects
            Rick Gerard Adobe Community Professional & MVP

            We need to know a little bit more about you project. Are these 144 video layers scaled? Are all layers visible at the same time? Are all of the video layers highly compressed MPEG streams from a digital camera? (Like a DSLR, or heaven forbid AVCHD?) Are you creating a video wall with 3-D layers? Very long render times would be expected if you stacked up 144 full rez 3-D layers in a video wall, pulled the camera back so you could see all of them, applied a light to them, cast shadows, and then did a camera move.

             

            If that's the description of your project, then the fastest thing to do would be to transcode your videos to a smaller frame size using a production codec that's easy to decode. Setting up a batch process would not be difficult to do. Then you would have a more reasonable number of pixels to manipulate.

             

            Think about this for a moment. 144 HD videos is a little under 900 million red green and blue pixels to run calculations on. With a better outline of your project, I should be able to, or someone else should be able to give you some suggestions for a better way, a more efficient way to manage your project.

            • 3. Re: AE CS5 rendering speeds with 100+ file sources, no effects
              mjjwatson2 Level 1

              Thanks for the replies.

               

              The videos are all mpeg streams, yes. There is no scaling, they are all 100% dimensions, no 3d. It is, in effect, a video wall, where only 120x120px segments of all the videos are seen, tiled. These segments lie side-by-side, creating a 16 x 9 grid of videos, hence the 144 original clips.

               

              They are all visible all the time.

               

              This is just the first step in a project- once I have created this wall of video segments, it will be used in a more fancy way. Although there's a large amount of videos, I didn't think there would be too much of an issue as there is no effects, no resizing, etc.

               

              Is it beneficial to convert the mpegs into something better for the production environment? Is there another way you can think of pulling this off?

              • 4. Re: AE CS5 rendering speeds with 100+ file sources, no effects
                Mylenium Most Valuable Participant

                You could pre-render all of them to image sequences. That way, at any given time AE only needs to look up the frame it actually needs without keeping a ton of clips open for file access. In fact had you considered this a bit earlier and also already downsized them for this purpose, you might be able to render this thing fast. On the other hand, it would have limited masking precision. Anyway, try this and use something like TGA or uncompressed TIFF. These formats are reasonably fast compared to PNG or compressed TIFFs. Just keep an eye on the colors. Multiple conversions may introduce Gamma shifts...

                 

                Mylenium

                • 5. Re: AE CS5 rendering speeds with 100+ file sources, no effects
                  Rick Gerard Adobe Community Professional & MVP

                  The MPEG is probably the biggest source of your slow down. Decoding all those mPEG frames into real frames AE can use takes a lot of horsepower. I would load your 144 videos into media encoder and then compress them to photo JPEG at the 95% quality. It shouldn't take too long if they're only five seconds long each. If all of your videos are longer than five seconds and you've trimed the in and out points, this could also be slowing things down.

                   

                  Is there some kind of reveal at the end? Animating the masks would also slow things down. Another thing I might try is to duplicate a 120 x 120 pixel solid and use that as a track matte. Solids are easy to align and distribute using the alignment tool. In some cases a track matte will render quicker than a mask.

                  • 6. Re: AE CS5 rendering speeds with 100+ file sources, no effects
                    mjjwatson22

                    Rick, Mylenium. Adobe seems to have made my account go rather weird, so apologies for not replying sooner.

                     

                    Had to make a new account, as you can see, but wanted to thank you two- by rendering all the mp4s into a lossless format it's saved the project, and render times are back to being sensible.

                     

                    Thanks again,

                     

                     

                    Michael