28 Replies Latest reply on Dec 4, 2016 5:25 AM by Gutter-Fish

    Particles taking too long to render

    mjshea

      My computer:

       

      Dell XPS laptop running Windows 10

      Intel Core i7 2670QM processor, 2.2GHz

      8GB RAM

      Nvidia GeForce 555M GPU

       

      I'm trying to render an AE project that is basically all particles using Particular. I have a null object with keyframes and I used expressions to trigger particle generation on those keyframes. I have probably 30 or 40 particle layers and most of them are generating particles more or less constantly. I tried to do a test render at half res (540p with full res at 1080p) for a 4 min project. It's been rendering for about 30 hours and estimated time remaining is at 150 hours. This seems a little excessive to me, but I have never worked with particles before. Is there anything I can do to cut down this render time? I have six of these videos and at this rate it will take months to render all of them in full quality, and that's just not going to fly.

        • 1. Re: Particles taking too long to render
          Rick Gerard Adobe Community Professional & MVP

          Render times for particle systems go up exponentially. There's no way around it except redesign and more creative use of the system.

           

          Try and simplify your design and think about generating separate particle streams that you can render and combine later. Whenever I use particular I try to limit the instances to three or four. With the right settings you can do an awful lot with three or four streams. I would never plan a project with 40 instances of particular in a single comp.

           

          Also, four minutes is an eternity in After Effects time. Most of my comps are under 10 seconds. Most shots in my finished work are less than five.  Unless there is something going on in the first frame that needs to continue all the way to the last frame of your four minute project consider breaking it up into shots or short sequences. You can then edit them together in Premiere Pro and have a lot healthier prohect in the long run.

          • 2. Re: Particles taking too long to render
            Szalam Adobe Community Professional & MVP

            40 particle layers?!?! That's...an unusually large amount.

            Do you have depth of field turned on?

            Do you have motion blur turned on?

             

            You have a tiny amount of RAM, so opening your project in CC 2014 to use multiprocessing to render won't likely offer much help.

             

            Any chance you could share a project file so one of us could poke through it and see if there's anything you could do to speed it up? It'd be a lot easier than guessing.

            • 3. Re: Particles taking too long to render
              mjshea Level 1

              I went back and counted and I actually have 54 instances of Particular.

               

              My project is a music visualization video. Each note is generating particles, which is why there are so many. I'm basing the animation off a midi file and then overlaying the mp3. Importing midi information to AE is a messy process. Most of the time I can have one instance of Particular per instrument (meaning there would be maybe 5 or 6 instances). But occasionally the midi information is imported in such a way that I have no choice but to have a separate layer for every pitch (meaning that, for example, I could have 15 layers for the piano alone if the piano plays 15 different notes throughout the song). As far as I can tell there is no way around this without compromising the accuracy of my visualizations.

               

              How would I go about sharing my project file?

              • 4. Re: Particles taking too long to render
                Rick Gerard Adobe Community Professional & MVP

                Personally, I think your concept is flawed. At the very least I would break up the project into four bar segments. Another option would be to put each note in a separate composition and then combine the rendered output. What's killing you render time this 50+ instances of particular trying to render at the same time. If you divide it up so two or three or four at the most or rendering it once and then you combine the output in an editing app like Premiere Pro you will be able to start building your project with some reasonable  and highly unlikely to crash in the middle of the render. Even if you could get 50 instances of particular to start rendering it is highly unlikely that you would get through the entire four-minute comp before something went wrong.

                 

                One other kind of side point, nobody, and I've been in music and video production for more than 40 years, can tell by looking at a visualization if the pitch matchd a stream of particles. Only about one in 100 can even can accurately tell if moving visuals are in sync with the beat. There can be a huge difference between visual perception and audio perception depending on how confusing the images are, the tempo of the music, and where your eye is in the screen when the downbeat happens.  Sometimes you need to lead to beat, sometimes you need to follow. It is very seldom that cutting exactly on the beat works for every shot.


                About 40 years ago I cut a very long piece to music and it looked perfect in my moviscope, and almost perfect on the flatbed editor, but by the time we  got the film printed and projected it at the dress rehearsal for the concert in the Kingdome in Seattle it looked like every cut was slightly out of sync. Matching the beat exactly to the cut gave most of us the impression that the music was late. Rather than recut the whole thing we just slipped the audio five frames and it looked great on the big screen. Just a little food for thought.

                • 5. Re: Particles taking too long to render
                  Szalam Adobe Community Professional & MVP

                  mjshea wrote:

                   

                  Each note is generating particles, which is why there are so many.

                  Totally not how I would do it. Instead of having an instance per note, tie the notes to the intensity of lights and use the lights as Emitters for 1 instance of Particular.

                   

                  mjshea wrote:

                   

                  How would I go about sharing my project file?

                  Upload it to your favorite cloud service (like Dropbox) and then share the link publicly if you don't care or privately if you're a bit more paranoid.

                  • 6. Re: Particles taking too long to render
                    mjshea Level 1

                    That is basically what I'm doing for most of the instruments. For any given instrument, say bass for example, I have 1 instance of particular in which the emitter position is tied to the pitch and the particles/sec is tied to the volume (so there are no particles if a note isn't being played). However the problems arise when the instrument plays more than one note at the same time. It's a flaw in the AE script that I'm using to import midi information. If there are multiple notes (multiple keyframes) at the same time, particular will only respond to one of them. So if I want particles for each note in a 3-note chord, I have to have 3 separate instances of particular in order for this to work. If everything worked like it's supposed to I would have no more than 5 or 6 instances of particular.

                     

                    I will try to upload my project tomorrow. Meanwhile is there anything I can do to improve the render time? I tried rendering as a JPG sequence instead of H.264 and that cut down the estimated render time from 200+ hours to around 50. But that's still a lot.

                    • 7. Re: Particles taking too long to render
                      Rick Gerard Adobe Community Professional & MVP

                      Render time goes up exponentially as you increase the number of instances of particular. As I said before the only way to cut your render time down considerably would be to limit the number of instances in each composition. You're trying to put everything in one composition and it is killing your render time. If you have one composition for the base, one composition for another instrument, and another composition each of the other instruments, all of those compositions rendered one at a time will take much less time than if all those layers were in one composition. That is just how the math and the calculations work for particle systems.

                      • 8. Re: Particles taking too long to render
                        Mylenium Most Valuable Participant

                        You're not listening to what the others are saying. Your whole approach is flawed in that Particular keeps evaluating even when it's not actually generating particles or only generating "invisible" particles and using expressions that are slow to evaluate doesn't improve matters, either. Also if your script generates a keyframe for every frame you would similarly have a baziliion of them that would influence evaluation performance. You are basically generating like 50 million particles or whatever at any given point or something like that, so with all due respect, how do you expect this to be faster? That wouldn't be trivial even in a much more sophisticated 3D program. And similar to Rick I find your claims to "precision" at least questionable. How many visible particles does a not actually generate for a few miliseconds? If you already have millions of particles, nobody will notice. In fact it would quite likely be visually distracting and annoying as indeed most people out there couldn't tell the difference between the individual notes and would wonder where that comes from. It's like designing light shows for concerts: If you accentuate every beat, you end up with a flickering lightning storm that causes epileptic seizures. I concur with the others: Your approach is simply flawed and you are obsessing over things that except your musician friends nobody will care about.

                         

                        Mylenium

                        • 9. Re: Particles taking too long to render
                          mjshea Level 1

                          It's not as visually complex as you're probably imagining it to be. This is pretty similar to what I'm doing: Original composition, midi visualization with processing - YouTube

                           

                          This was done in processing and not AE, which probably makes a lot more sense for something like this, but I don't know processing.

                          • 10. Re: Particles taking too long to render
                            Rick Gerard Adobe Community Professional & MVP

                            Let me make this simpler for you with some numbers. Say one instance of particular takes 1 second per frame, and 2 instances take 2 seconds per frame then you would expect that 3 instances would take 3 seconds per frame. This isn't the case, 3 instances take  4 seconds per frame, 4 take 8 seconds per frame and 5 take 16, 6 take 32, 7 take 64, 8 takes 128 and so on. The numbers don't work exactly like this but this is the basic principal of particle rendering. This means that 50 streams running at the same time will have pretty insane system requirements and render times.

                             

                            On the other hand, if you take say 8 streams and break them up into comps that contain 2 streams each then you have a total of 4 comps to render each of which takes 2 seconds per frame. Your render time is 2 seconds per frame for 4 comps for a total of 8 seconds per frame for all comps combined. That's a whole lot more efficient than 128 seconds per frame for 8 streams in one comp.

                             

                            Let's talk a bit about your expressions now. When you are writing expressions that evaluate time it is very easy to write an expression that is recursive. What that means is that the expression looks at previous frames by including a time function so that on frame 1, frame one is evaluated but for frame 20 frame one through 20 must be evaluated. Stack a bunch of recursive expressions up and you'll also exhaust system resources and slow things down very quickly.

                             

                            The example you gave kind of looks like a piano roll with glow linked to volume. If you are trying to do something similar using particular for 50 or so notes I would break up each few notes into a separate comp that was just big enough to contain the particles for those notes and render them all out separately then combine the 10 or so sections into a final master project in either AE or Premiere Pro. You'll get the same effect with some reasonable expectations of render time. The only thing that this approach to the project requires is that you can visualize the final effect with your designers eye before you see it.  Putting 50 instances of Particular in a single comp, even if the number of particles and the physics involved are small and throwing in expressions that could easily be running in recursive loops will kill your system very quickly.

                             

                            Without at least a sketch of what you are trying to do it's hard to visualize the most efficient way to achieve your design goal. Post a screenshot of what you are trying to do and maybe we can help you. The 50 instances of Particular just is not going to work.

                            • 11. Re: Particles taking too long to render
                              Szalam Adobe Community Professional & MVP

                              mjshea wrote:

                               

                              I tried rendering as a JPG sequence instead of H.264 and that cut down the estimated render time from 200+ hours to around 50. But that's still a lot.

                              Ooh, you shouldn't ever render an H.264 directly from AE anyway, so that's a good step to take.

                               

                              It might help if you got more RAM 8 GB is, like, the MINIMUM I would suggest for basic AE work. You are not doing basic AE work. But Rick's suggestion of breaking your comp up into multiple comps is good if you can't do my suggestion of using lights as emitters and having only one instance of Particular.

                              • 12. Re: Particles taking too long to render
                                mjshea Level 1

                                I didn't realize that's how particular worked. I will try breaking it up into smaller comps like you suggest. Right now my "instruments" are pre-comped, so all the guitar layers are in a pre-comp and all the piano layers are in a pre-comp etc. Would it achieve the same effect if I render each pre-comp individually then stack them back together in AE?

                                • 13. Re: Particles taking too long to render
                                  mjshea Level 1

                                  Unfortunately getting more RAM is not an option at this point. I know the system I have is far from ideal for AE work but I'm stuck with it for now.

                                   

                                  Would you mind walking me through the method you're suggesting of using lights as emitters? How is it different from using 1 instance of particular with pitch tied to emitter position and particless/sec tied to volume?

                                  • 14. Re: Particles taking too long to render
                                    mjshea Level 1

                                    Here's a screenshot from my project.

                                     

                                    Dropbox - Get Lucky_01437.jpg

                                     

                                    The horizontal blue lines are vocal harmonies that move across right to left with time. The red at the bottom is bass, the yellow vertical lines are guitar, and the blue "puffs" are piano notes.

                                    • 15. Re: Particles taking too long to render
                                      Rick Gerard Adobe Community Professional & MVP

                                      It's right there in the help files: Emitter> Light Emitter | Red Giant

                                       

                                      I would strongly suggest you go through the Trapcode  section in the Red Giant site and look at some tutorials on Particular. You could easily spend an entire semester in college studying this one Effect. Trying to figure it out by looking at a couple of amateur YouTube videos or worse yet just poking around is a little nuts.

                                      • 16. Re: Particles taking too long to render
                                        mjshea Level 1

                                        So I'm rendering only 4 instances of particular at a time, and it's going much faster. However, it seems like the rendering slows down over time. When I started it a couple hours ago it was rendering several frames per second, and now it's about 2/3 of the way through but it's only rendering one frame every few seconds. Maybe this is a newbie question but do you guys know what's causing this? In past projects my render speed has been pretty consistent throughout, granted those projects never involved particles.

                                        • 17. Re: Particles taking too long to render
                                          Szalam Adobe Community Professional & MVP

                                          In the beginning, there aren't many particles. As the process goes along, you have more and more particles. The more particles you have, the longer it's going to take. If you can, make the lifespan of particles shorter. If you can, make fewer larger particles as opposed to many tiny ones.

                                          • 18. Re: Particles taking too long to render
                                            Rick Gerard Adobe Community Professional & MVP

                                            You could also have a recursive expression in your project. They can be killer. A recursive expression retrieves values from previous frames using time functions.

                                             

                                            We could tell what is going on if you included a screenshot of your particular layer with all modified properties revealed. Just select the layer, one that has an expression, and press the U key twice, then take a screenshot (Alt + print screen) then paste it to your desktop, then drag the screenshot to the reply field in this forum so we can see what's going on.

                                            • 19. Re: Particles taking too long to render
                                              mjshea Level 1

                                              I have two different variations that I use depending on the instrument. Here's a screenshot of the expressions I'm using for those.

                                              screenshot1.pngscreenshot2.png

                                              • 20. Re: Particles taking too long to render
                                                mjshea Level 1

                                                There's actually one more. What I'm trying to achieve with this one is to create a burst of particles whenever it hits a keyframe. Is there a way to do this without using recursive expressions?screenshot3.png

                                                • 21. Re: Particles taking too long to render
                                                  Szalam Adobe Community Professional & MVP

                                                  If you can convert keyframes to layer markers (pretty sure there's a script out there for this), you can trigger animations based on layer markers (see this: Light Leaks at Markers - CreativeCOW).

                                                  • 22. Re: Particles taking too long to render
                                                    mjshea Level 1

                                                    I was able to convert the keyframes to layer markers, but I'm still not able to trigger a burst of particles on those markers. The Creative Cow link you posted is a preset with a limited number of options. As far as I can tell it doesn't allow you to trigger a custom animation or a particle burst.

                                                     

                                                    I thought it might work if I used this expression with particular set to "explode" instead of "continuous"screenshot1.png

                                                    But that results in a burst of particles on the first keyframe and nothing after that.

                                                     

                                                    Do you have any other suggestions? It's simple enough to create a constant stream of particles that "switches off" when the volume (v) is zero. That's what the above expression does. But what I want to do is create a single burst of particles triggered either by a keyframe or a marker. As I showed in my previous post, I figured out how to make this work using recursive expressions. But is there any other way that won't slow down my rendering so much?

                                                    • 23. Re: Particles taking too long to render
                                                      Rick Gerard Adobe Community Professional & MVP

                                                      Since you are trying to animate the birth rate of particles I think the best approach would be to add a light for each particle stream you want to use and then tie the birth rate to your slider controls. I'm assuming that the values for the slider controls would give you a curved line rather than just an on and off. You could then use the linear interpolation method to set the threshold for your particles and control the number. For example I set particular up to use lights named Emitter 1 and Emitter 2 as the emitters and used the light intensity as a modifier for the birth rate. Here's the expression:

                                                      t = thisComp.layer("Audio Amplitude").effect("Both Channels")("Slider");

                                                      linear(t, 10, 16, 0, 500)

                                                      When I checked the graph for the Both Channels slider that was generated by the Keyframe Assistant I determined that I wanted values between 10 and 16 to trigger particles. The light intensity is a percentage multiplier for the birth rate so if you set the minimum value to 0 you get no particles when the slider value is below 10 and when the slider value is 16 you get 5 times the birth rate for the emitter. This does not have to look forward or backward at keyframes and it runs very fast.

                                                       

                                                      If your slider is either on or off like a switch and has no decay then you can buy a little more time by using easeOut instead of linear and jack the number of particles way up so a bunch are born in an instant. Here's a simple example. I have used this for muzzle flashes, for fireworks, and a bunch of other things and it doesn't slow down with 10 or 15 lights. The lights are also 3D so if you add a camera to the scene you can control the size of the particles by the distance from the camera.

                                                       

                                                      Here's a quick example. I clipped the input values so the audio works more like a switch than anything else. You can see the dying particles and the new burst from the threshold being crossed a couple of frames before the CTI. This is probably the approach I would have taken with your project in the first place:

                                                      particular.jpg

                                                      • 24. Re: Particles taking too long to render
                                                        mjshea Level 1

                                                        I'm probably misunderstanding what you're saying, but I tried using easeOut and it still results in a stream of particles. Instead of a blue stream like I have here, I just want a blue "puff" every time I hit a key frame.screenshot4.png

                                                        • 25. Re: Particles taking too long to render
                                                          Rick Gerard Adobe Community Professional & MVP

                                                          You have hold keyframes. That means the graph editor is going to show you square shapes. The trick to making this work is to turn particle birth rate to zero and have the lifetime and speed set so that you get breaks in the particle stream. Under the CTI the value is 250, If you move the CTI to the other side of the keyframe does the value drop to 0? I would not have chosen hold keyframes because it would be nice to have the number of particles reduce as time passes. You can still get results with square corners but it may be harder to see the empty spots. The easiest way to see what's going on is to use the graph editor and make sure you have turned on Show Post Expression Graph. I built something like this:

                                                          particular 2.jpg

                                                          Changing to hold keyframes you get a different look but there will be a constant number of particles and this may make the empty spots more difficult to see when you are creating smoke.  Check the Particles Per second with the graph editor. If your graph shows periods of no particles being generated and there is no gap the lifetime is too long. You might also want to adjust the opacity over lifetime so the first particles fade out as they reach the end of their life. This graph shows particles per second and you can see what I did to opacity over lifetime.

                                                          Screen Shot 2016-04-06 at 5.41.57 PM.png

                                                          If you used lights as emitters and colored the particles with light color then you could put ten streams together that look like this without much of a hit in rendering time. Here's a quick test with 7 different colored lights all tied to the same slider. There was about a two or three second increase in render time over the previous version with just one stream.

                                                          multiple lights.jpg

                                                          I hope this helps.

                                                          • 26. Re: Particles taking too long to render
                                                            mjshea Level 1

                                                            Yes, this is very helpful. This is my first time working with Particular so I appreciate your patience with my lack of knowledge.

                                                             

                                                            I removed all time-based expressions from particular and played around with the keyframes in the graph editor, and I was able to achieve an effect pretty close to what I want. This is what my graph editor looks like. It's one slider on a null object containing all the midi information, and the particles/sec is pickwhipped directly to the slider.

                                                            screenshot5.png

                                                            I was hoping that removing the time-based expressions would solve the render slowdown, but it didn't. Even with those expressions removed, it still takes MUCH longer to render frames near the end of the comp compared to the beginning. Do you have any more suggestions on how to troubleshoot this?

                                                             

                                                            EDIT: I just went back and tried scrubbing through on a different instance of particular that had no time-based expressions to begin with, and that one also takes significantly longer to render frames near the end of the comp. So I guess the time-based expressions are not the culprit here? And it's not the number of particles or anything because that's pretty consistent from beginning to end. Any ideas what could be causing this?

                                                            • 27. Re: Particles taking too long to render
                                                              claudiakunin Level 1

                                                              I am having the same problem......what used to take 20 hours to render is now taking a week!

                                                              WTF?

                                                              • 28. Re: Particles taking too long to render
                                                                Gutter-Fish Level 4

                                                                claudiakunin wrote:

                                                                WTF?

                                                                Maybe there's a better way to ask a question.

                                                                FAQ: What information should I provide when asking a question on this forum?