22 Replies Latest reply on Jun 9, 2017 2:23 PM by Roei Tzoref

    Why is the grain filter so slow?

    nullsebasvideo Level 1

      The "Add Grain" filter performs so poorly that I'm wondering if it was written in the 90s and never touched again. On a very short composition with a few other adjustment layers, doing a RAM preview takes about 20 seconds. If I enable the adjustment layer that has the grain filter, each frame takes like 3 or 4 seconds and the whole composition takes about 3 minutes to render. This is on an i7 3930k with 32 GB of RAM which flies on anything else. Looking at the task manager, when it's rendering it doesn't even get to 10% CPU, so I'm guessing it's a very poorly coded filter, unless it's using only the GPU, but when anything uses the GPU I can feel its fan revving up, and it's not the case here. This is with CS6 by the way.

        • 1. Re: Why is the grain filter so slow?
          Rick Gerard Adobe Community Professional & MVP

          Just a lot of temporal calculation going on here.

          • 2. Re: Why is the grain filter so slow?
            Mylenium Most Valuable Participant

            The "Add Grain" filter performs so poorly that I'm wondering if it was written in the 90s and never touched again.

             

            Well, kinda... It was a third-party product once and Adobe bought/ licensed it. Just like all these plug-ins and features (Keylight, Timewarp as well etc.) it may not get the attention that would be necessary due to the specifics of licensing deal not allowing certain changes or other such stuff. That and of course as per Rick's comment all temporal calculations in AE tend to slow down things considerably...

             

            Mylenium

            • 3. Re: Why is the grain filter so slow?
              nullsebasvideo Level 1

              Mylenium wrote:

              and of course as per Rick's comment all temporal calculations in AE tend to slow down things considerably...

              Whatever temporal calculation it needs to do, if it used 100% of the processing capability it wouldn't take so long. This is supposed to be a professional product, not some open source beta.

              • 4. Re: Why is the grain filter so slow?
                Rick Gerard Adobe Community Professional & MVP

                a frame of video is only so big. Once you've loaded all of the data from a frame in and performed the temporal calculations you've used all of the processing power that can be used until you read in another frame. This turns into a question of bus speed and not CPU power. Until the basic architechure of the plug-in and the way AE reads and writes frames to memory changes temporal processing will be slow and not capable of utilizing 100% of the processor power. While AE is far from perfect it's the most powerful compositing and effects app for the money by far. Always has been.

                • 5. Re: Why is the grain filter so slow?
                  Mylenium Most Valuable Participant

                  This is supposed to be a professional product, not some open source beta.

                   

                  After Effects has been in permanent Beta for a few years now, considering how bugs and other issues are piling up with every new version... Otherwise it comes to the old same limitations that have always existed in the program: Temporal operations require linear progression and thus are handled step-by-step in a single thread. That's why they are so slow. If you need better performance, you can always buy commercial third-party plug-ins that sidestep the issue one way or another...

                   

                  Mylenium

                  • 6. Re: Why is the grain filter so slow?
                    nullsebasvideo Level 1

                    Mylenium wrote:

                     

                    This is supposed to be a professional product, not some open source beta.

                     

                    After Effects has been in permanent Beta for a few years now, considering how bugs and other issues are piling up with every new version... Otherwise it comes to the old same limitations that have always existed in the program: Temporal operations require linear progression and thus are handled step-by-step in a single thread. That's why they are so slow. If you need better performance, you can always buy commercial third-party plug-ins that sidestep the issue one way or another...

                     

                    Mylenium

                     

                    That's the problem with After Effects, sometimes it seems like it's merely a hub for 3rd party filters that are better than the ones it bundles. While 3rd party effects are a good thing in some cases, you shouldn't have to spend extra money in a plugin just to have film style grain. Instead, I added an adjustment layer, threw in noise and a gaussian blur, adjusted it and I can't tell the difference. Obviously it renders much faster.

                    • 7. Re: Why is the grain filter so slow?
                      Rick Gerard Adobe Community Professional & MVP

                      Grain is slow because it simulates grain. Less in the highlights, more in the shadows, just like film. 3rd party plug-ins became a lot of the bundled plug-ins but that does not stop other developers from building their own better mousetraps. Noise and gaussian blur doesn't look like film grain, it looks like noise. Maybe it's good enough for you and it's important to know when good enough is good enough.

                       

                      It wasn't too long ago that it took several million dollars worth of equipment to do what AE can now do on your desktop. Unless this is just a hobby for you, there's nothing keeping you from charging your clients enough for your services to stay current, use the plug-ins required to do the job efficiently, and make a profit. If you are in business and are not doing that then be prepared to go broke. If it's a hobby work some overtime or spend the time needed with AE to get the job done with what you can afford and be glad you still don't have to spend more than twice what you paid for your home for your first setup to edit SD video on a desktop. I did that in 1994, but it wasn't a hobby, it was my business.

                      1 person found this helpful
                      • 8. Re: Why is the grain filter so slow?
                        Todd_Kopriva Level 8

                        > sometimes it seems like it's merely a hub for 3rd party filters that are better than the ones it bundles

                         

                        We deliberately make After Effects a platform on which others can build. That's a good thing.

                         

                        If we dedicated all of the resources that we'd need to do a better job natively than every specialized plug-in does for its own area, we'd need vastly more people working on the software, it would become much larger, and you'd have to pay a lot more for it.

                         

                        If there's something that every After Effects user needs, then we try to build that into the application. If it's something that only some need, we may build something basic into the application (like the 3D Camera Tracker) and leave a third party to do the more full-blown version. If it's something that few people need, we make sure that our plug-in and scripting APIs are good and we work with other folks who want to build the extensions themselves.

                         

                        Regarding the grain effects specifically: We get very few requests to make them faster. If you want them faster, then you can ask by telling us here: http://www.adobe.com/go/wish

                        • 9. Re: Why is the grain filter so slow?
                          nullsebasvideo Level 1

                          Todd, I see your point, but some things shouldn't be so subpar. A film grain effect is something basic, I shouldn't have to buy a 3rd party plugin just to get that, or wait several more times to render a frame just because I want to use it. How hard can it be to optimize the same filter to utilize all the CPU power, or at least most of it?

                          • 10. Re: Why is the grain filter so slow?
                            Rick Gerard Adobe Community Professional & MVP

                            How hard can it be to optimize the same filter to utilize all the CPU power, or at least most of it?

                            It would take a complete rewriting of the underlying operation of AE. AE processes a pixel at a time and performs mathematical calculations on each pixel in the frame and combines those calculations with each pixel in the frame of the next layer. It does this a frame at a time.

                             

                            If you have a 2K frame with 10 layers then that's 20K of data that needs to be processed. If it's temporal effects that are looking a frame a head and a frame back then you're stuck with 20 K of data calculations a frame at a time added to the additional 20 K of data from the next or last frame. That's not very much work for 16 CPU's to do. It's why AE doesn't run at 100% when some 3D apps can. Totally different math threads.

                             

                            The third party solutions for grain that work better still have these limitations, but they have just come up with faster or better ways to make the same calculations that are more efficient.

                             

                            Asking AE to use 100% of the processor power and 100% of the memory is like asking a 10 inch water main to stay completely full when you've only got a gallon of water in the system and you can only access it through a 1/2 inch pipe.

                             

                            I'm not making excuses, just trying to explain how AE works. Can Adobe make it better? Sure, for the most part they have with every release that I"ve used in the last 20 years. If you have a good knowledge of how AE processes the pixels then it will help you make workflow and production decisions that make your work more efficient and profitable. It's all part of being a pro.

                             

                            BTW, Flame, Inferno, all of the other big compositing apps have the same limitation. Only when rendering something more than pixel math does the system start to max out... Try AE with a particle system like Particular and 20,000,000 particles and you'll see CPU and Memory use go up.

                            • 11. Re: Why is the grain filter so slow?
                              Mylenium Most Valuable Participant

                              you shouldn't have to spend extra money in a plugin just to have film style grain.

                               

                              Depends. I never use grain in my technical animations, so I'd already consider this a specialized use case. Occasionally I do use Remove Grain to enhance undersampled Ambient Occlusion stuff and that...Sure, it could be faster, but it's okay. I can always have anotehr cup of tea when it renders. It's mostly a problem while working and tweaking parameters...

                               

                              Mylenium

                              • 12. Re: Why is the grain filter so slow?
                                nullsebasvideo Level 1

                                Rick Gerard wrote:

                                Asking AE to use 100% of the processor power and 100% of the memory is like asking a 10 inch water main to stay completely full when you've only got a gallon of water in the system and you can only access it through a 1/2 inch pipe.

                                 

                                Well, in that case Adobe should spend a little time getting at least a 5 inch pipe. I'd be happy if at least it used 50% of the CPU, it would mean not waiting 10 or 20 times extra render time, especially when having 5 other adjustment layers renders in a tiny fraction of a second.

                                • 13. Re: Why is the grain filter so slow?
                                  Todd_Kopriva Level 8

                                  As I said before...

                                   

                                  Regarding the grain effects specifically: We get very few requests to make them faster. If you want them faster, then you can ask by telling us here: http://www.adobe.com/go/wish

                                   

                                  We work on things largely based on how much they're requested. This request hasn't come up enough for us to dedicate our limited resources to it. I suspect that this is because there are various third-party alternatives, ranging from inexpensive and good to expensive and excellent.

                                  • 14. Re: Why is the grain filter so slow?
                                    Navarro Parker Level 3

                                    If grain is really really important to you and you don't like After Effect's performance, try round tripping your footage via dynamic link through Premiere and Speed Grade.

                                    • 15. Re: Why is the grain filter so slow?
                                      altarvic Level 1

                                      Hi,

                                      You can use film grain overlays.

                                      They provide real-time rendering, unlike film grain filters. And the most important - these plates were created from real film stocks.

                                      1 person found this helpful
                                      • 16. Re: Why is the grain filter so slow?
                                        nullsebasvideo Level 1

                                        altarvic wrote:

                                         

                                        Hi,

                                        You can use film grain overlays.

                                        They provide real-time rendering, unlike film grain filters. And the most important - these plates were created from real film stocks.

                                         

                                        My point is that After Effects should not only include this grain filter as it does, but make it at least a bit efficient so it doesn't slow down rendering to a crawl. What you link to here is a paid product. I already paid for After Effects, it should bring a decent film grain filter, that's a given in any compositing program.

                                        • 17. Re: Why is the grain filter so slow?
                                          Dave LaRonde Level 6

                                          Have you tried imposing a film grain in other applications?  An editing application, perhaps?  Smoke of Flame?  Something else?  Can you describe your experiences with them?

                                          • 18. Re: Why is the grain filter so slow?
                                            nullsebasvideo Level 1

                                            Dave LaRonde wrote:

                                             

                                            Have you tried imposing a film grain in other applications?  An editing application, perhaps?  Smoke of Flame?  Something else?  Can you describe your experiences with them?

                                             

                                            I don't have or ever used Smoke or Flame. As far as editing apps go, I can tell you this: Edius has a filter called "Old Movie/Film" which has all the typical stuff such a filter requires, included Film Grain, and it plays real time without even rendering. This is on Edius 6, a 32 bit app, and not aided by the GPU in any way. Edius is proof to me that when programmers do their best, they can produce a video app that doesn't need the latest computer or the latest graphics card or even 64 bits. Edius 6 is the fastest and most stable software available, one that never crashes and has real time so good that it's like editing from tape, even on processor intensive formats.

                                             

                                            But back to the film grain filter in After Effects, I just don't use it. I use the simple noise filter with about 0.3 Gaussian Blur and I can't really tell the difference, and I don't have to wait forever for it to render. The film grain filter is useless unless you really need a specific film stock, but if all you need is some film like grain, what I do is good enough.

                                            • 19. Re: Why is the grain filter so slow?
                                              lukaszbanach Level 1

                                              Yes it is really slow, there is no discuss about that. There is a lot of plugins for CC2014 that are using gpu and are like 20 times faster.

                                              Grain Removal and Add Grain are the slowest plugins in whole app, even slower than very complicated plugins like 3d elements performing faster.

                                               

                                              I don't understand, why Rick Gerard is defending something which is clearly poorly written.

                                              1 person found this helpful
                                              • 20. Re: Why is the grain filter so slow?
                                                alvanylander Level 1

                                                Isn't possible to create your own film grain overlays? i can't find anyone that has written about that but I don't see any downsides and it's free?

                                                 

                                                Shouldn't you just be able to export a seperate composition with a solid that has 50% brightness 0% saturation with an "Add grain" filter applied that has the settings you want. 

                                                 

                                                If you add that footage to your original composition it should be able to render much faster, no?

                                                 

                                                Is there a quality loss or something?

                                                • 21. Re: Why is the grain filter so slow?
                                                  Dave LaRonde Level 6

                                                  There are freebie grain clips out there, and if you're not picky about the precise nature of the grain, they'll do the trick. 

                                                  I work in a business where "here's your project -- have it done by the close of business today" happens a lot, so I tend to use such shortcuts a lot.

                                                  • 22. Re: Why is the grain filter so slow?
                                                    Roei Tzoref Adobe Community Professional
                                                    Isn't possible to create your own film grain overlays?

                                                     

                                                    it certanley is

                                                     

                                                    Shouldn't you just be able to export a seperate composition with a solid that has 50% brightness 0% saturation with an "Add grain" filter applied that has the settings you want.

                                                    yes. more precisely, create a complete grey layer RGB 127 127 127 with grain, render a few seconds in a lossless format, reimport it and set it in overlay mode above your footage.

                                                     

                                                    If you add that footage to your original composition it should be able to render much faster, no?

                                                    yep

                                                    1 person found this helpful