21 Replies Latest reply on Apr 17, 2016 7:18 AM by Rick Gerard

    Prevewing is slower when project is large

    guyman20

      Hello

      I'm working on a very largepojrect (a template I bought), but I'm only using a small portion of it in my working composition.

      However, I opened a software that tracks files that are being read/written from/to disk, and I can clearly see that, while previewing, AE is using source files which do not appear in my composition and are completely redundant for this preview!

      This probably slows down the preview a lot.

      Is there any explanation for that?

        • 1. Re: Prevewing is slower when project is large
          Dave LaRonde Level 6

          Yeah, you need to remove the extraneous footage from the project.  You'll need to find it in all those nested comps in the template.  Since templates tend to be VERY complicated, with no two alike, you're on your own to find them.

          • 2. Re: Prevewing is slower when project is large
            Rick Gerard Adobe Community Professional & MVP

            After Effects looks at everything in a comp and decides what to do with the pixels. If you have a great big layer that is scaled way down AE looks at the great big layer first. It's important that your assets are sized appropriately so that at some time in a composition they are at the equivalent of 100% scale. This means a 3D layer at 100% scale that is the same distance from the camera as the zoom value. If you have a huge image that is only visible to the camera in 1/16 of the frame because it's out of position AE still looks at the whole image before it starts the calculations. Most of the work is just done with assets in the Composition Panel's view and Active Camera is the only view that will render.

             

            There are also assets that just live in the Project Panel that can take up a bunch of resources, especially if they are highly compressed video files or long image sequences. If you bought a template and you've whittled it down to what you are going to use and the system is really slow I would suggest that you save a copy of the project then select the comp you plan to render and then go to the file menu and choose File>Dependencies and then choose Consolidate Files to build a folder with just the used assets and generate a new comp or File>Dependencies>Reduce Project or >Remove Unused footage. This will clear out everything that is not directly related to the main comp.

             

            There is one other thing you might try. If there is a comp that is complicated and contains none of the assets you are adding to the comp then you might want to render a lossless with alpha copy of that comp and replace it  in your main comp. I'm rendering and replacing all the time with complex projects so that I can make deadlines that would be missed if I stubbornly tried to work with a comp that had an unusually high overhead. In the long run you'll save time and preserve your sanity as you develop efficient workflows.

            • 3. Re: Prevewing is slower when project is large
              guyman20 Level 1

              Hi Rick and Dave

              Dave - I can't remove footage from the project, because I can't know which footage I'm going to use and which not.

              Rick - rendering layers won't help, because once I'm finished working on a layer - I move to the next few seconds.

              If I work with a few layers, I can't really know which one is ready in order to render it. I'm constantly changing all layers until I think the segment is ready. At this point - there's no point of rendering this segment, because I'm not going to view it too much while working.

              also, as I wrote to Dave - reducing the project can't help, because I can't know when and if I'm going to use unused footage.

               

              Any other suggestions?

              Thank you

              • 4. Re: Prevewing is slower when project is large
                Rick Gerard Adobe Community Professional & MVP

                First, AE is for working with shots so if you have a huge project then you should do at least the rough editing in Premiere Pro to figure out what shots are working and how you want to do the project. I routinely work on shots that are 4 to 10 seconds long in AE that are part of larger sequences. AE is for the visual effects part and not for editing. You do that later. Some of my shots (comps) have a hundred or more layers and things can slow way down.

                 

                If you have purged your cache then none of the assets that are just sitting in the Project Panel will effect the render or the preview. You said you bought a template but are only using a part of the template. I'm not sure what you mean by that. If the template is 5 minutes long and you are only using 10 seconds of the template then set a work area for the part you are using, check out the layers in the comp and the nested comps and delete the one ones that don't matter, then trim your comp to the work area. It's not completely necessary but it might save you a little bit of resources because AE only considers the frame it's working on and anything outside the timeline isn't part of the calculations.

                 

                I have never worked on any comp that I couldn't separate out a problematic bit and get it right, then render and replace that part in the bigger project. I do it all the time. This is especially important if you are working on something that you know is going to tie up a bunch of system resources like Warp Stabilizing a shot. Warp stabilizing is a real drain on the system and it stays a drain even after it's done. If I have a shot that is more than a couple of seconds long that requires this kind of processing I always render a DI (digital intermediate) and replace the footage in the project then delete the warp stabilized comp completely. This is done with 3D renders, temporal effects like time blending and a bunch of other procedures that tie up system resources and dramatically drive up render time when you combine them with other effects.

                 

                You say you can't remove footage from the project because you can't know what footage you are going to use. Well I'm sorry but you have to decide what you are going to work on. It's impossible to get anything done any other way. If you have a ton of footage and you can't decide what shot to use then you need to pre-edit something in Premiere Pro so you can at least have an idea where you are going or you'll never get anything done. Pick the parts of the sequence that you want to work on and do quick tests to see if they help you tell your story. There is no need to load every possible asset you could possibly use in the entire film into one project file. If you have several shots that you want to try to apply the same basic effects on then the efficient way to do that is to create a comp with one of the shots, add the effects, then duplicate the comp in the Project Panel and replace the footage. If that shot requires tracking or other timing changes then its far more efficient to pre-edit and visualize without the effects than it is to do the same thing 4 or 5 times. If you are trying to make a living at this then that's the fastest way I know of to go broke.

                 

                Creating a movie is all about making decisions. Maybe a short story will help. I worked on a project a bunch of years ago that involved 6 months of shooting about 20 different people six or seven days a week with two cameras in San Francisco's China Town and the rural town of Locke, CA . The project was to be shown on public television as a two hour program 30 days after the last day of shooting. We ended up with about 600 hours of footage that had to be cut down to 2 hours. We got it done by reviewing the footage and the days events every single day of filming and taking notes. As the story developed we started to divide the story up into segments and discuss how to handle each of the different parts. We started the editing by throwing out a bunch of good stuff because it didn't contribute to the story we wanted to tell and then rough cutting each segment we thought we would use as a separate project. Each of these one to five minute segments (scenes) were reviewed and we tried to decide how effectively the segment contributed to the story. Then we cut together all of the edited segments into the final project to see how the story worked. The first cut was about 30 minutes too long so we threw out some sections completely and then cleaned up, trimmed, and polished each of the segments in their original editing project to make them more effective. The changes rippled down to the show master and with a couple of more tweaks we were ready 5 days before the air date. The project was shot and edited on film so I won't go into the part about managing reels and saving the trimmed frames even if hey were only 5 frames long, but that is what we had to do back then.


                That's exactly how they work on feature films and that is how it has been done for more than 100 years. You decide, trim, polish the pieces, see how it works, then you polish the pieces again. You don't start on frame one and go the last frame in a single timeline, you build something that you can easily go back into scene 7 and change a shot to improve the product.

                1 person found this helpful
                • 5. Re: Prevewing is slower when project is large
                  guyman20 Level 1

                  Rick, thank you for your detailed response.

                  However, I don't think your solutions can be applied on my case.

                  A template is not of a constant length.

                  A template is a bunch of compositions, graphics and effects that can be used in various ways.

                  When opening a template, you get all of the graphics in it, and you can't reduce the project before you finish working on it - because you don't know which graphics you're going to use.

                  I don't think that the size of the AE file should influence the RAM it uses. I don't understand why AE keeps reading files that I don't use.

                  Another thing is that the RAM AE takes, is only getting bigger and bigger and never cleans up until I restart AE (even if I purge).

                  Is there a memory leak?

                  • 6. Re: Prevewing is slower when project is large
                    Dave LaRonde Level 6

                    I don't think you understand templates made for AE.  They're almost always for a single purpose, with very few parameters you can change unless you REALLY know what you're doing.  You can change their length, but not by much.  99 times out of 100, you just make your tweaks according to the instructions, render and you're done.  They're not cornucopias of footage and graphics.

                     

                    But to answer your slow previewing lament, we don't know a thing about this template.  We don't know the effects it uses, if it's 2D/3D, the nunber of lights if any... there are all sorts of things that affect the time it takes to render a frame for preview.  And I don't think we've nailed down what you mean by the term, "previews slowly".  For all we know, it could vary from "not instantaneous" to "a week".

                    • 7. Re: Prevewing is slower when project is large
                      guyman20 Level 1

                      Hi Dave LaRonde

                      First, regarding the template -

                      The one I use is indeed built the way I described it, here it is:

                      Pixity Land | Character Explainer Toolkit - After Effects Project Files

                       

                      But let's leave that aside for a moment. I do understand the lack of information in this post that doesn't allow to fully understand the situation, so let's focus on one general problem -

                      What I learned in the last few days is that, regardless of the speed of previewing (which we'll leave that aside for now) - I think AE is behaving as if it leaks memory-

                      After a decent use of 1-2 hours and some smooth and long previewing- I'm starting to experience a decrease in the length of the previews, until AE is only able to render just a few frames.

                      When I look at the RAM, I do see that it's being used at almost 95% when the previews are short (as opposed to about %60 at the first few previewing).

                      Purging the memory doesn't help.

                      Restarting AE does solve this issue.

                      It looks like the previews that are rendered are kept in RAM until it "explodes", which is indeed a memory leak.

                       

                      I'm considering buying new hardware, but if that's really the case theneven 64GB RAM would be eaten up after some time.

                      Is there anything we can do to check that?

                      Thank you!

                      • 8. Re: Prevewing is slower when project is large
                        Szalam Adobe Community Professional & MVP

                        guyman20 wrote:

                         

                        What I learned in the last few days is that, regardless of the speed of previewing (which we'll leave that aside for now) - I think AE is behaving as if it leaks memory-

                        After a decent use of 1-2 hours and some smooth and long previewing- I'm starting to experience a decrease in the length of the previews, until AE is only able to render just a few frames.

                        When I look at the RAM, I do see that it's being used at almost 95% when the previews are short (as opposed to about %60 at the first few previewing).

                        Purging the memory doesn't help.

                        Restarting AE does solve this issue.

                        It looks like the previews that are rendered are kept in RAM until it "explodes", which is indeed a memory leak.

                        What exact version number of After Effects are you running?

                        • 9. Re: Prevewing is slower when project is large
                          guyman20 Level 1

                          AE CC 2015 13.7.1.6

                          • 10. Re: Prevewing is slower when project is large
                            Szalam Adobe Community Professional & MVP

                            That sounds like a pretty bad issue, but it's one that I haven't experienced and haven't really heard anyone else with it either. This is good because it means it's probably something that can be isolated fairly easily. I'd recommend filing a bug report about this issue and be as specific as you possibly can about everything particularly the contents of your compositions when you get into that state.

                            Feature Request/Bug Report Form

                            • 11. Re: Prevewing is slower when project is large
                              Dave LaRonde Level 6

                              Got everything you need in your project?  Save the project under a new name.  Get rid of the unnecessary stuff in this new project.  Use the Reduce Project command.  Use the Collect files command -- it'll copy just the stuff you need to a new location, including the project file.  The original project file, plus all the other footage, remain unharmed.

                               

                              Now begin work again.  If you STILL have previewing problems, we're going to need a whole bunch of additional information to even have a fighting chance at a diagnosis:

                              FAQ: What information should I provide when asking a question on this forum?

                              • 12. Re: Prevewing is slower when project is large
                                guyman20 Level 1

                                Dave, as I explained above - I'm using a tempalte which holds all the possible graphics I might want to use.

                                Only when I finish creating video I know which graphics I use and which not, so while working I can never know which graphics I'll need and which not.

                                What else do you recommend?

                                Is what I described considered as a typical behavior?

                                • 13. Re: Prevewing is slower when project is large
                                  Dave LaRonde Level 6

                                  Sounds like you need to start making choices about the stuff you WILL use and get rid of the rest.  A storyboard will help.  Know what a storyboard is?  Start drawing.

                                  • 14. Re: Prevewing is slower when project is large
                                    guyman20 Level 1

                                    That sounds like a patch that will hide a real problem behind it.

                                    As a programmer in my profession, I can certainly say there is no reason the RAM will become full the longer I use the program, especially not because files sitting in the project that I'm not using (causing previews to get shorter and shorter).

                                    Is there any real technical solution that won't require me to pre-choose hundreds out of thousands of files before I start working?

                                    • 15. Re: Prevewing is slower when project is large
                                      Dave LaRonde Level 6

                                      One last thing I can offer is that the OS is messing up the application..... but as I recall, you haven't said what it is.  Down to the last decimal point.  I'm a bit surprised you haven't mentioned it.

                                       

                                      You also have other version options besides the latest and supposedly greatest version of AE.  You can download and install AE CC 2014 to see if it works better.  Same thing goes for AE CS6.

                                       

                                      How to Find and Install Previous Version of Adobe Apps in CC 2015 | Adobe Customer Care Team

                                       

                                      If your template will run on an earlier version, I'd give it a shot.  You might not have to weed anything out.  But then, weeding things out comes with the video production territory.  You've got to make choices on what to keep and what to toss all the time.

                                      • 16. Re: Prevewing is slower when project is large
                                        guyman20 Level 1

                                        It actually looks like the problem also occurs on a previous version of AE (2014).

                                         

                                        Dave, I'm running Window7 64bit, 16GB RAM, 120GB SSD and 1TB of hard drive.

                                        I've set my assets to sit on the SSD and thecacheto sit on hard drive, and figured it's better than doing the opposite (As I wrote, for some reason - AE is constantly reading the assets, so it's faster to place them onSSD).

                                        I don't have a paging file set on SSD, only on my hard drive (Manually set to 6000MB). It might be the problem?

                                        What other settings should we consider in this case?

                                         

                                        Previews are faster now that the assets are on SSD, but I can definitely recreate the memory leak -

                                        If I run a few previews one after another, they're starting to get shorter and shorter, until I can no longer preview.

                                        It looks like AE is not clearing the previous previews from RAM when it's out of space.

                                        Purging won't help.

                                        I'm wondering if someone experiences the same problem.

                                        Are you always able to generate previews and are never forced to restart AE?

                                        I'm afraid that if I buy more RAM - the problem will remain and I'll just be able to hold a bit longer before needing to restart AE.

                                        • 17. Re: Prevewing is slower when project is large
                                          Dave LaRonde Level 6

                                          Since it happens on two different AE versions -- which are quite different under the hood, incidentally -- I don't think the problem can be laid at Adobe's doorstep this time.  But before I raced out to get new RAM, I'd do a painstaking check of the hardware.

                                          • 18. Re: Prevewing is slower when project is large
                                            guyman20 Level 1

                                            Dave, do you think the problem might be the hardware? I've never seen such issues other than when using AE.

                                            How do you recommend checking the RAM?

                                            • 19. Re: Prevewing is slower when project is large
                                              Dave LaRonde Level 6

                                              You can do it in the OS.  At least you can on my Win 7 box at home.  A complete hardware check.

                                               

                                              And as my IT Lady buddy told me, "The cheaper your memory chips, the greater the chances they'll go bad."

                                               

                                              So if you got your RAM from Sergei's Sacrificial Clearance Computer Emporium.ru  for a song, you might be paying for it now.

                                              • 20. Re: Prevewing is slower when project is large
                                                guyman20 Level 1

                                                Dave, are you talking about this test?

                                                It's in Hebrew, but I translated the key words, in red.

                                                scr.png

                                                 

                                                As you can see, my computer is in top condition. I bought it from a store who specializes in computers for software editing, and paid top money for it.

                                                I don't believe the problem is with the hardware.

                                                 

                                                I can totally replicate the bug.

                                                Does anybody want to volunteer replicating it on his system? I would send the AE project, all you have to do is run a few previews until you start seeing they're getting shorter and shorter, as your RAM fills up and never gets emptied.

                                                 

                                                Thank you

                                                • 21. Re: Prevewing is slower when project is large
                                                  Rick Gerard Adobe Community Professional & MVP

                                                  Rendering time is based mainly on the things that are going on in the composition. Some projects just take a long time to render. Folks new to compositing and effects are often surprised by the time it takes to render. I routinely have effects that take 4 or 5 minutes per frame to render. Sometimes I build comps that render at 10 frames per second or faster. It all depends on what you are doing. The only real way to tell if your system is running slower than it should because of some hardware or software problem is to carefully analyze the processes that are running and compare them with a similar machine that is rendering significantly faster. Common speed killers are Anti Virus programs, poor system configuration, third party apps running in the background, lack of free disk space, and network congestion. Common user error problems that cause huge rendering slowdowns are improper sizing of the resources used int he comps (images that are way too big), poorly organized temporal effects, poorly thought out time stretching or time blending, excessive samples for motion blur, and improper configuration of the AE preferences. If you want help with the preferences then search for Optimizing After Effects and read the suggestions offered by Adobe. If you think your workflow may be inefficient then describe it in detail and look for all modified properties in all layers in the comp. If you are using Ray-traced rendering make sure that you really need it because that will completely kill your rendering speed, may not work at all if you use the Adobe Media Encoder, and anything you can do with Ray-traced rendering can be done more efficiently using C4D Lite and Chinaware that now come with AE.