7 Replies Latest reply on Oct 27, 2017 1:08 PM by DanTull

    Final Release - RAM management?

    donjojo-hannes Level 1



      I have to make a decision that depends on any possible improvements on RAM management (and the release date):


      WHERE I'M AT:

      I have a 15 episode (6 hour) project lined up to be finished. Puppets, rigging, background footage, camera angles, drawings, explainer graphics/animations are all finished. All I need to do now is record the performances and replace the static placeholders I used to setup the other elements of the scenes in AE through dynamic linking and hit export.



      My episodes are long. The puppets are 4k to make different shots possible. There are hefty 4k image cycles etc. In short: as described a while back, my 64 Gig of RAM get used up and eventually the system crashes, because - I assume - the current beta is not part of the Adobe CC max RAM settings in the preferences. So every puppet calls up an instance of CH and the longer the scene, the more RAM each instance uses, until - if long and complicated enough - the RAM maxes out and the system crahes.



      1) Exporting from CH to a 4K file with transparency and working with it in AE

           cons: -     any changes require a new export (which may well take 5 hours)

                     -     huge files: storage requirements     

      2) Splitting the scenes.

           cons: -      That is painful. Because I have 15 episodes in 6 languages and it becomes messy quickly

                     -      I still need an intermediary export (from AE) and patch things together in Premiere



      Will Character Animator CC be part of the CC Suite in the preference settings, so that instead of hogging more and more ram it respects the limits defined there and uses a (flash) drive as virtual memory extension? Yes this will slow it down, but it is better than crashing.

      Or will this not work because of how dynamic linking operates in general? Or will the next release simply lack this improvement for other reasons?


      An answer to this question will help me greatly in deciding if it makes at all sense to delay the release.



      If RAM management improves: the only deal breaker would be if the release date is not reasonably soon. The update video spoke of "this fall". That would be now-ish. I hate to go through all the pain of the work arounds and then find out that I would have had just to wait 2 more weeks for an easy cruise.
      So, while I understand you will not give me a date, I would be happy if someone from the Staff could just (even privately) message a là "given the pain of the work arounds, I would wait and take walks under the colored leafs for a little while" or "man, do your stuff now. What is done is done. Suffering builds character. Plus, if you do it now, you will have more time to bake cookies for Christmas later."




        • 1. Re: Final Release - RAM management?
          oksamurai Employee Moderator

          I asked the team and got this:


          "Short answer is no, we don't participate in the Memory management preference dialog.  We would need to track down where that memory is being used and whether it is something we can dynamically purge during execution.  We do keep an art cache for each Ch scene, but typically these don't use that much memory. Either of his suggestions should allow him to finish his project."




          "Wow, definitely never pushed the app that hard in the time or size scale. I don't even think I have a machine that I could even get close to surviving such a project. All my machines _combined_ don't have 64 GB of RAM, much less the compute horsepower it'd take to not make rendering it an exercise in pain (and possibly fire safety).


          So basically it sounds like the length and size are so large that you're running into the limitations of the application. I'll DM you for some more particulars.

          • 2. Re: Final Release - RAM management?
            DanTull Adobe Employee

            Hi Johannes --


            I'd be curious to see a screenshot of Activity Monitor/Task Manager showing the number of Character Animator and other high memory processes and how much each one is using. Also, do you find that there are more processes than you expect based on the number of Character Animator scenes you are linking into After Effects at any given time?


            This is definitely past the scale I've pushed Character Animator before, but want to make sure I understand the dimensions of how you're pushing it when trying to understand what might be accumulating so I can confirm the issue at a smaller scale.


            Thanks -- Dan Tull


            • 3. Re: Final Release - RAM management?
              donjojo-hannes Level 1

              Thanks Dave for checking with the team. It is not the answer I was hoping for, but the answer does help me in that I now know I have to work around the issue now rather than wait.


              More in response to Dan, below

              • 4. Re: Final Release - RAM management?
                donjojo-hannes Level 1

                Hi Dan,


                If I had faster Internet I could share project files etc, but as it is, I'm literally sitting on a mountain (well, the hut is on a mountain. I'm sitting on a chair, in that hut) and data packages are brought by eagles or something. :-P




                If you want, I can attach a screenshot of the task manager (I would need to "destabelize" my system first), but the screenshot might not indicate a problem. The instances of Character Animator that run as part of dynamic linking in the background seem normal. There is one instance for each CH scene used in the AE project. That makes sense.

                In those two scenes in CH I have a total 3 puppets (4k puppets, between 150 and 180 mb each with cycle layers and shabang - file size tweaked to be as small as I can get them). 1 puppet hat its own scene. The other 2 share one scene.


                I bring those two scenes into a 1080p AE project where a total of 5 camera angles provide medium shots of each scene and a close up of each character. All those angles make for a dialoge setup, so there are frequent cuts between the views. I set up all the other elements with dummies and color bgs instead of video footage (just dialogue audio) because editing that way is naturally much faster and means, I only need to replace the dummies and bgs before the render.
                This is what my timeline looks like:





                To make things fun for my system my scenes have real life 1080p footage in the background, which due to the camera setup has some nice blurry action going on. Blury-ness both affects the video and CH scenes slightly due to the nature of cameras in AE.  That certainly explains why AE swallows up some processing power and Ram. But the processor has never used full power I think and memory usage on the AE part is not critical.




                The length of the episodes can be as long as 40 minutes. When working on the first few (time line) minutes of the project in AE, scrubbing works and previews happen. But the more I advance in the timeline, the more RAM gets used by the 2 instances of character animator (if character animator happened to be simultaniously open as a program, that 3rd instance is idle at some low RAM usage). As I understand, this happens, because through dynamic linking all the info up to that particular point in the AE timeline needs to be stored in RAM. Early in the timeline = less RAM; later in the timeline = more RAM. When I had 32 gig of RAM I actually crashed the machine a few times. After the upgrade I barely managed to get to 27 minutes. It reaches 95-98% RAM much earlier and with a few prayers it survives and bit by bit renders and frees up things if there is no sudden demand by the OS that could lead to a max out I guess. That is why I believe that if RAM was managed through the global preferences, the system might remain stable and get through the render.

                Currently I leave 80% of RAM to adobe products (which is plenty on a 64gig system). I guess I could really curb AE and force it to use the flash drive, so even more of the RAM is free for CH, but I suppose it would just give be a few more render minutes. As it is, I think the two instances of CH already would both hog as much as 45 gig.


                So if RAM in Character animator will not be managed through the global adobe CC preferences even in the new release, then splitting the episodes is the only way to go for me. It is painful because it adds another render step (and I have several language versions coming up) but if this is the only way, it is the only way.



                • 5. Re: Final Release - RAM management?
                  DanTull Adobe Employee

                  Got it, thanks for the details. That gives me enough to do some experiments and try to see what's being retained to cause that sort of memory accumulation.



                  • 6. Re: Final Release - RAM management?
                    donjojo-hannes Level 1

                    Thanks Dan. Happy hunting!

                    • 7. Re: Final Release - RAM management?
                      DanTull Adobe Employee

                      Almost forgot to come back and explain what's up here.


                      When another app (AME, After Effects, etc) uses Character Animator to serve frames over dynamic link, it's important that when they ask for the same frame again, we produce the same output. Since Character Animator has complex animation state over time, to do that can require re-running the simulation from the beginning, so we cache the information (but not the pixels) required to regenerate those frames. For complex puppets that can get quite large. If the framerate is high or the scene is long, it adds up.


                      It'll take some work, but we have some ideas on how trade off random access performance with memory usage by either discarding older frames and accepting the need to regenerate or writing that data to disk instead of keeping it all in memory. Sorry that won't happen in time to save you the trouble on this project, but maybe the sequel!


                      Thanks for the report!




                      P.S. I like the style of those characters. I'm looking forward to seeing this when you finish!