10 Replies Latest reply on Sep 3, 2010 2:22 AM by phil__ramses

    AE Frustrates me a lot on a Dual Xeon. Need Help.

    phil__ramses

      Hello,

       

      I just got a new comp to work intensively onto After Effects CS5 :

       

      -Dual Xeon X5680
      -36 Go ram

      -1 SSD win 7 pro 64b

      -2x Raid0 Data in / data out.

       

      The first thing I did was to re-run a project I had rendered on my previous computer and who took 5 hours to renders.

      So I set MP on, left 3go for the other application and set 24x1Go (for each thread). Render. OMG. estimated time : 7 minutes left...

       

      Taskbar say 100% cpu business, and nearly 20-33 go ram occupation.

       

      And then suddenly, after like... quarter of the render is done, the cpu business just felt to 5%, Ram business to maybe 5go, and estimated time raised to more than 3 hours.

       

      I abort the render, and, surprise, a sort of error popup saying :

       

      "After Effects warning: A frame failed to render while using Render Multiple Frames Simutaneously. Allocating more memory to the background processes in Memory & Multiprocessing Preferences may fix this problem. (26 :: 142)"

       

      I did. Started again. And yeah... It did 20 frames more.

      I tried a lot of adjustement. Nothing solved.

       

      So I started "Googling" and found this :

       

      "Note: When the RAM cache (the RAM available for storing RAM preview frames) is nearly full, the background processes cease rendering and go into a low-memory-usage state, and the foreground process starts rendering—just as if Render Multiple Frames Simultaneously were off. This means that the rendering of a RAM preview will be fast at first, and then the speed will step down as rendering switches to only one processor core." here

       

      The thing is :
      -I've set & enabled all the cache folder, enabled disk cache to it's max (500000 MB), and it doesn't matter.
      -If I run my render, interrupt it when it goes down, and the re-launch where it ended, again and again, I do my render in like 10mn overall

       

      MY QUESTIONS ARE:

       

      -Is there a way to set the amount of RAM Cache?

       

      -Is there any way to purge it automatically?

       

      -Am I forced to stay along my computer to watch/abort/relaunch my renders to take benefit of my configuration?

      If the answers are NO,NO,YES, well, are adobe engineers/leader projects dumbs? Because that will be a SERIOUS shame to deliver a such no-professional solution.


      -And, okay, in the worst case, considering that my computer have 2x6 cores, and the fact that AE recognize 24 threads, how much memory should I get to avoid this ****?

       

       

      Thanks for reading & advices.

        • 1. Re: Ram cache question on a very strong rig
          Mylenium Most Valuable Participant
          how much memory should I get to avoid this ****?

           

          Whatever your motherboard can handle, which should be 96GB on most of them.... The rest, including your insults, is beside the point. There's enough guidance on that in the online help:

           

          The optimum amount of RAM to allocate for each of the background processes varies according to project settings (such as color bit depth), composition settings (such as pixel dimensions of the composition frame), and what effects are applied. For a typical standard-definition television project, at least 1 GB per background process is recommended. For a typical project with HDTV-sized compositions, at least 2 GB per background process is recommended for optimum performance. Start with these settings, but run some tests with your own computer systems and projects to determine the best settings for your specific needs. Digital cinema projects and projects with larger frame sizes require even more RAM per background process for optimum performance.

           

          With all respect, you can't blame anyone, you just didn't read and completely misconfigured your system, so the behavior is not the result of some dumb engineer not checking his math. In the meantime, simply use less cores and ramp up the per CPU memory settings. And of course feel free to use the purge every n-th frame option in the "secret" prefs (hold down Shift while entering the General prefs page). Keep in mind though, that this may not help at all - if the footage still needs to be loaded for effects processing, there is no point in purging it for every frame. just adds file I/O overhead that may eat up all processing benefits.

           

          Mylenium

          1 person found this helpful
          • 2. Re: Ram cache question on a very strong rig
            phil__ramses Level 1

            Thanks Myllenium.

             

            And going to "insult" (I just suggested some lack of smartness) was just the result of my big frustration.
            Anyway, I persist by saying that a option or a behaviour should allow AE to just reset the Ram cache already used... and rendered.

             

            "-If I run my render, interrupt it when it goes down, and the re-launch  where it ended, again and again, I do my render in like 10mn overall"
            That should be automatic for god sake. AE should be abble to manage that by it self, don't you think? Is that idea so incredible?

             

            You have to know that I'm a big fan of AE. Really.
            But the software isn't perfect, and there is still big improvements to do, especially in the memory handling.

             

            So If I get your advice & Logic, I should get AT LEAST 48Go dedicated to AE to use 24 threads, right?
            I'm not sure that will help. I did some test with 12 threads enabled using 2Gb per core, that didn't solved. The renders went farer, but not till the end.
            And getting 96Go will not help myself at all


            Anyway, this morning I saw that was now possible to allow 3Gb per core, I will try that.

             

            And finaly I forgot to mention, but I did some test with enabling the secret menu, and that was just worse than without purging

            • 3. Re: Ram cache question on a very strong rig
              Mylenium Most Valuable Participant

              If you provide more info, I'm sure we can figure it out. This may in fact be a memory leak in a CoDec due to the kind of footage you are using. Also check your Temp directory. MP rendering produces a lot of temporary files. if the directory is full and no files can be added, this may also nuke the MP rendering...

               

              Mylenium

              1 person found this helpful
              • 4. Re: Ram cache question on a very strong rig
                phil__ramses Level 1

                Thanks again Myllenium.

                 

                I do my render in Tiff sequence. And I will check my Temp directory, but I think that as far you have free space on C:\, the temp folder can be filled in.

                Anyway, I will check that tonight, and will see.

                 

                About the other informations, I will post everything that can be usefull.

                 

                Regards.

                • 5. Re: Ram cache question on a very strong rig
                  phil__ramses Level 1

                  I did some research and it seems than the Temp folder doesn't not have a maximum size.
                  Anyway, I found that I could increase the maximum virtual memory. I gonna try this and report if it helped or not.

                  • 6. Re: AE Frustrates me a lot on a Dual Xeon. Need Help.
                    phil__ramses Level 1

                    Okay I did some tries yesterday :

                     

                    I tried using 24 cores allocating 1gb per core.
                    I tried using 12 cores, allocating 2gb per core.
                    I tried using 8 cores, allocating 3gb per core.
                    I tried using 4 cores, allocating 3gb per core.

                    I tried using 2 cores, allocating 3gb per core.
                    As long i reduce the core & increase the ram per core, the render just go slower and does longer, but still goes in "one core" mode at a time.

                     

                    If I run AE as Admin, it just freeze when I start MP render...

                     

                    I added windows virtual memory to other disks, letting windows managing it.
                    I cleaned up my cache, my temp folder, and so on all the stuff. Anything help.

                     

                    The really strange stuff is : That never happened to my old Q6600 with 8go of ram.

                    The other strange stuff is that while I was trying & tweaking AE preferences, at the end, AE was just not abble to render one single frame before going to "throttle mode". So i deleted my preferences file... and... that changed anything. So now, I can't even reproduce my initial conditions

                     

                    The other thing who's strange is:

                     

                    "Note: When the RAM cache (the  RAM available for storing RAM preview frames) is nearly full, the  background processes cease rendering and go into a low-memory-usage  state, and the foreground process starts rendering—just as if  Render Multiple Frames Simultaneously were off. This means that the  rendering of a RAM preview will be fast at first, and then the speed will step down as rendering switches to only one processor core."

                    That refers to RAM PREVIEW, not real rendering. Meaning my real render behave like a Ram preview (is that normal?).

                     

                    I really need help, because it really doesn't looks like a normal behaviour, and the more I goes, the less I'm sure that adding ram will solve or really help...

                    • 7. Re: AE Frustrates me a lot on a Dual Xeon. Need Help.
                      Mylenium Most Valuable Participant

                      So what footage and effects are actualyl involved? You still haven't told us any of that. As I said, this may be a leaky CoDec or some effect doing weird things because they are not MP safe or generally have a problem with current multicore CPUs due to wrong compiler settings having been used to produce them (or outdated programming methodology).

                       

                      Mylenium

                      • 8. Re: AE Frustrates me a lot on a Dual Xeon. Need Help.
                        phil__ramses Level 1

                        Thanks Myllenium.

                         

                        And I already answered what was the output .
                        I actually use any footage. The composition is filled with around 12 layers of particular v2. And I'm rendering it as a Tiff sequence (no movie).

                         

                        I really have no more clue of what it could be. Anyway, it still bug me that on my old computer, I never met this situation

                        • 9. Re: AE Frustrates me a lot on a Dual Xeon. Need Help.
                          Mylenium Most Valuable Participant

                          So it's all synthetic? In that case you should have no issues. However, since you mention Particular 2, you may see a normal state wher due to more and more particles being gnerated it eats up all your memory. This would in particular be true - no pun intended - with custom particles due to their shading features and possibly temporal sampling of clips accumulating a lot of data. I would look into that.

                           

                          Mylenium

                          • 10. Re: AE Frustrates me a lot on a Dual Xeon. Need Help.
                            phil__ramses Level 1

                            I did some tries yesterday again, this time with composition having variety of footage & fx, as rasterized swf, glow, 3d with light & shadow & motion blur...
                            With that, my cpu business grewd up nearly 90-95% having 6 cores with 3go allocated.
                            And, Jesus, it goes fast as hell. My old quad is a baby's toy .

                             

                            So, yes, I really suspect that my first issue is more related to some lack of optimisation in multithreading & memory management from particular. Sad.

                             

                            I will now need to check with longer composition, but looking at the difference of the behaviour, I've got something normal now, so...

                             

                            Thanks for your time & interrest Mylenium