34 Replies Latest reply: Jan 26, 2015 12:26 AM by adobefishman RSS

    Saving Projects with Warp Stabilized Footage takes Forever

    AtonMusic Community Member

      Hi,

       

      1)

      I have 8 comps in my project all enjoying the wonderful stabilization of Warp Stabilizer.

      When I save the project - it takes 10 minutes. Removing all the Warp Stabilizers from the footage it takes 1 second.

       

      2)

      Having done all my work in the 8 comps all having Warpy applied to them then dragging the comps over to Adobe Media Encoder for rendering takes about 20 minutes.

       

      Removing Warp Stabilizer from the Comps and repeating above takes 1 minute.

       

       

       

       

      I think that warp stabilizer is great but then workflow slowness hit is devastating.

       

      Is all this normal behavior ?

       

       

      Thanks

        • 1. Re: Saving Projects with Warp Stabilized Footage takes Forever
          Todd_Kopriva Adobe Employee

          What are your Memory & Multiprocessor preference settings?

          • 2. Re: Saving Projects with Warp Stabilized Footage takes Forever
            Todd_Kopriva Adobe Employee

            Also, what are the characteristics of your compositions---most importantly, duration and frame dimensions?

            • 3. Re: Saving Projects with Warp Stabilized Footage takes Forever
              Dave LaRonde Community Member

              Such a long save may be abnormal, but you should know that ALL of the tracking information and settings for each piece of stabilized footage is saved in the effects themselves: no keyframes to speak of, but tons of data. 

               

              I'll bet you would find that there is HUGE difference in project file size between a project with stabilized footage and and identical project with no stabilization.

              • 4. Re: Saving Projects with Warp Stabilized Footage takes Forever
                AtonMusic Community Member

                Hi Todd,

                 

                thanks for chiming in...

                 

                Memory installed = 32Gb / 28GB  set for other apps (No other apps are running)

                Processors enabled = 4 CPUs with 2GB Per CPU assigned

                Project = 1920x1080 tiff Sequence 300 frames long.

                 

                Warp Stabilizer set to synthetic edges

                 

                After Effect is quit prior to AME rendering

                AE Project is 125MB and YES it is smaller when no warp stab. is applied.

                 

                 

                I think it has to do with the fact that AE assigns more than 100% load to ONE CPU.

                And AFAIK the Maximum ONE cpu can reach is 100% not 160% which AE assigns them to use.

                At least ONE core is running at 160% when viewed in OS X Activity Monitor.

                 

                All in all - I fare a whole lot better just by leaving MulitProcessing off.

                 

                Perhaps in CS7 - the leverage of MP is developed enough.

                Apple probably plays a huge part in this too. I know of many applications having problems with MP'ing.

                 

                Aside from Apple's own apps which are using every bit of CPU power there is and really are excelling on it

                • 5. Re: Saving Projects with Warp Stabilized Footage takes Forever
                  AtonMusic Community Member

                  Hi Dave,

                   

                   

                  depend on how one regards HUGE..

                   

                  THe project is about 3MB without Warpy and 125 with it...

                   

                  Still saving a 125 MB should not take so long!

                   

                   

                  But what do I know.

                   

                  Still, warp stabilizer is a god darn amazing stabilizer. Really incredible for 90% of all tasks.

                  And in those 90% there are NO stabilization app available that comes even close !

                   

                  I ran some tests against a lot of the others and while on long pans warp stabilizer is just as 'bad" as all the others; when it comes to roll-shutter stabilization and general jitter-removal it makes me smile every time I use it...

                   

                  Thanks Adobe !

                  • 6. Re: Saving Projects with Warp Stabilized Footage takes Forever
                    Dave LaRonde Community Member

                    AtonMusic wrote:

                     

                    THe project is about 3MB without Warpy and 125 with it...

                     

                    Still saving a 125 MB should not take so long!

                     

                    To me, that's a HUGE difference between 3 and 125! 

                    And I think I might have an explanation for some of your slowness.

                     

                    You previously wrote, " Memory installed = 32Gb / 28GB  set for other apps (No other apps are running)"

                    ...which would be 4GB of memory allocated to AE.

                     

                    You also wrote, "Processors enabled = 4 CPUs with 2GB Per CPU assigned"

                    ...ah, but 4 x 2GB = 8GB! 

                     

                    Are you unnecessarily forcing AE to use disk caching?

                     

                    I would think you easily could allocate 16 GB to AE, and 4GB to each processor AE uses.

                    • 7. Re: Saving Projects with Warp Stabilized Footage takes Forever
                      AtonMusic Community Member

                      Dave,

                       

                      I think you missed in my other post the part where I stated that AE isn't even running while I encode in AME !

                      • 8. Re: Saving Projects with Warp Stabilized Footage takes Forever
                        Dave LaRonde Community Member

                        I'm not talking about the encoding, I'm talking about your sluggish saving woes within AE itself.  I thought that was the real problem.

                         

                        I think you need to adjust AE's processor and memory allocations; they don't look right, and I think that at their current settings, you're forcing AE to do disk caching to run on the amount of memory you have allocated to it... which I think is causing the slow save times.

                         

                        Have you tried the recommended settings outlined here before?

                        http://blogs.adobe.com/toddkopriva/2010/10/please-try-recommended-memory-settings-for-afte r-effects-cs5-and-give-feedback.html

                        • 9. Re: Saving Projects with Warp Stabilized Footage takes Forever
                          AtonMusic Community Member

                          Hey Dave,

                           

                          thanks for taking the time to write...

                           

                          Sorry.. yeah, the real problem was the save time but I think it is intertwined with Drag n Drop to AME !

                           

                           

                          Anyway, at NO time is my memory maxed out during that process. I have a monitor showing me what memory is being used and what not.

                           

                          But I will try out todd's suggestion which you pointed me to via the link. Then I'll report back !

                           

                           

                          Thanks again !

                          • 10. Re: Saving Projects with Warp Stabilized Footage takes Forever
                            Todd_Kopriva Adobe Employee

                            > Sorry.. yeah, the real problem was the save time but I think it is intertwined with Drag n Drop to AME !

                             

                             

                            They are related. When you process an After Effects composition in AME, a headless version of After Effects starts up and works in the background. It has to load the project and all of the other stuff that After Effects would normally have to do.

                            • 11. Re: Saving Projects with Warp Stabilized Footage takes Forever
                              AtonMusic Community Member

                              Opening the same project takes 10 minutes too....

                               

                              Now, I am kind of guy that like saving all the time, I have that command+S - built-in to my brain.

                               

                              I just cant wait 10 minutes EVERY time I hit command+save each 5 minutes.

                               

                              Great stabilization or not... That time-lag isn't worth it.

                               

                               

                              I have tried with ALL MP settings recommend. While AE does faster/slower RAM-Previews and Renders depending on the MP settings - the MP settings have absolutely NO effect on my saving/opening problems.

                               

                               

                               

                              todd, is this something you guys have been experiencing too.

                               

                               

                               

                               

                              PS - If Warp Stabilizer is causing the slow open/save due to its data being altered, perhaps I could understand that LAG.

                              However, if absolutely nothing has changed in its data then I dont see why it AE should save it all over again !

                               

                               

                              PPS - Draggin over 10 comps to AME from AE (same project) takes more than half an hour.... If AME only has to open the AE project ONCE then why does it need so long as if it seems that it would open the project 10 times consecutively !

                              • 12. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                AtonMusic Community Member

                                Found a MUCH faster way of doing all this !

                                 

                                I save my AE project, then close it. Then in AME I select File>Add AE Composition...

                                 

                                It will take about 4 minutes for Dyn Link to find the Comps in the project but once so, I can just select all 10 comps and add them in ONE SECOND.

                                 

                                That is GREAT news. Dragging from AE takes more than 30 Minutes to reach the exact same result !

                                • 13. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                  Todd_Kopriva Adobe Employee

                                  Thanks for reporting back. That is useful information for us to isolate what is taking so long. I'll pass this on the the relevant engineers.

                                  • 14. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                    Paul Uusitalo Community Member

                                    Something doesn't seem right here. While it is true that the Warp Stabilizer does significantly increase project size, and cause a certain level of slowdown in project save/load times, ten minutes seems seriously out of line for a 125 MB project. I just constructed a similar project here - over 250 MB in size and it saves in just under a minute.

                                     

                                    I'm wondering if something is amiss with your project file. And you are correct, memory and MP settings really have no affect on save /load performance.

                                     

                                    Could you possibly email me a copy of your project?

                                     

                                    puusital@adobe.com

                                     

                                    Thanks!

                                     

                                    Paul Uusitalo

                                    After Effects Quality Engineering

                                    • 15. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                      AtonMusic Community Member

                                      Hi Paul,

                                       

                                       

                                      unfortunately, this was what I call a temporary project. Once the Comps have undergone rendering the project is deleted.

                                       

                                      Next time I come across this void, I'll be sure to email you are project - using the email address you gave me !

                                       

                                       

                                      Thanks for responding !

                                       

                                       

                                      I might add that ALL the comps had synthetic edges set to 5 seconds.

                                       

                                      PS | Last findings before I deleted the projects was that I noted that setting the Synth.Edge value to 0.5  made the project save faster. Setting the method to Stabilize only made it save even faster!

                                      • 16. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                        Paul Uusitalo Community Member

                                        Yes, if you create another project that exhibits the same behavior, please let me know.

                                        • 17. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                          Bonsai Gravy Community Member

                                          Howdy!

                                           

                                          Hope you don't mind if I piggyback on this thread but I just ran into the same problem.  For me, projects saved in After Effects CS5 usually run around 1 MB; a similar project in a trail version* of AECS5.5 (with Warp Stabilization) recently ran at 61.5 MB.  Seems a bit too extreme, though it does explain why it takes so long to save and open projects.

                                           

                                          Rendering CS5.5 is also a time hog.  Each time I hit the preview button I get a series of prompts in the info panel beginning  "Background Process Status, Loading Projects" which takes around 3 minutes to execute, before the very slow rendering process begins. 

                                           

                                          Having never used WS before (and I do love it!) I began to layer several affected clips, which resulted in extremely slow rendering times.  Then I rendered the warped-stabilized clips and layered the render clips, but that also resulted in long preview times.

                                           

                                          Let me throw this in, too: on several occasions I rendered items in the render queue, waited during the "Saving Project" prompt and 3 minute spinning rainbow wheel, but the result was a QT mov of one only second.  Then I would add the same clip back into the render queue and get the result I wanted.  Perhaps I'm drinking too much coffee, or I just belong to the "I need in done yesterday!" generation, but the whole process seems to be taking a lot more time than it should.

                                           

                                          *I am using the trial version.  Is this something that gets cleared up when I install the full version?  I am awaiting it's arrival as we speak.

                                           

                                          Here's my particluars:

                                           

                                          MacPro3,1 running Snow Leopard (10.6.8)

                                          Quad-Core Intel Xeon

                                          Processor Speed: 3 GHz

                                          Dual processor (3GHz)

                                          Total Number Of Cores: 8

                                          16 GB RAM

                                          Using a chart generated for an Adobe article by Todd, my memory and multiprocessing settings are as follows:
                                          Memory: 6GB for other apps
                                          MP: 3GB RAM allocation per background CPU
                                                 Installed CPUs: 8
                                                 Actual CPUs that will be used: 2
                                                 CPUs reserved for other apps: 4
                                          Thanks in advance.

                                          • 18. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                            Ken Bit Community Member

                                            Hello all

                                            may i add some Bits to this discussion.

                                            i have recently upgraded, camera to nikon d5100 and up to CS5.5 and have been amazed at the power of warp stabilize, especially for extreme and almost abstarct use (when invoking the synthasize function).  yet as with Aton asbove i have quickly manged to get mega and unweildy file sizes once ws has been added. when i queue a whole bunch of clips, 20 plus i have had 20 30 minute save times. as i wanted to process the whole project this work flow should not be an issues, yet it quickluy becomes unworkable.

                                            also i seem to have run into a killer issues in which render queues will fail or are perhaps becoming corrupted.

                                            i have taken a particular clipthat has failed and put it into its own project, applied the same adjustments and WS and rendered the proect seemlessly. yet when isolating the same file in the 20 plus render queue it fails once again.

                                             

                                            i guess i can work around this by more or less treating each clip individually, but with  shoots that have dozens of smallish clips this too is unweildy.

                                             

                                            im guessing there has to be a better way of storing the WS information than in the project file in its current format, such long save times are obviously unstable i various ways.

                                             

                                            any further discussion on this would be most welcome.

                                             

                                            best best

                                            Ken

                                            • 20. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                              Dave LaRonde Community Member

                                              Mr. Zogg wrote:

                                               

                                              Save Project.jpg

                                               

                                              That's too bad.  Perhaps you might help us help you:

                                              http://forums.adobe.com/thread/961743

                                              • 21. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                                Mr. Zogg Community Member

                                                Nope Dave, that doesn't help a bit.

                                                • 22. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                                  Paul Uusitalo Community Member

                                                  Can you give some specific details as to your Project, System, other processes running, etc.?

                                                   

                                                  Not too much to go on here with just a screen shot.

                                                   

                                                  Thanks!

                                                   

                                                  Paul Uusitalo

                                                  After Effects Quality Engineering

                                                  • 23. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                                    Dave LaRonde Community Member

                                                    Mr. Zogg wrote:

                                                     

                                                    Nope Dave, that doesn't help a bit.

                                                    Sorry to hear that you don't find it useful to follow a link to a post by Adobe's Todd Kopriva entitled, "What Information should I provide when asking a question on this forum?"

                                                     

                                                    Yours must be a profound problem indeed.

                                                    • 24. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                                      mikeo55555 Community Member

                                                      I've had this problem for over a year now, but now I am starting to stabilize GoPro3 4K footage in AE CS5.5 and saves are taking SOOOO Long !

                                                       

                                                      has there been any fixes ? I don't care about the render times but the saves are killing me. I can't upgrade to CS6 due to work limitations.

                                                       

                                                      CS 5.5

                                                      Mac OS 10.6.8

                                                      Model Name:    Mac Pro

                                                      Model Identifier:    MacPro4,1

                                                      Processor Name:    Quad-Core Intel Xeon

                                                      Processor Speed:    2.26 GHz

                                                      Number Of Processors:    2

                                                      Total Number Of Cores:    8

                                                      L2 Cache (per core):    256 KB

                                                      L3 Cache (per processor):    8 MB

                                                      Memory:    12 GB

                                                      • 25. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                                        Paul Uusitalo Community Member

                                                        How many frames are you stabilizing in a given project?

                                                        • 26. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                                          Rick Gerard CommunityMVP

                                                          Why not render the stabilized footage? I never leave stabilized footage in a comp. I don't see the point.

                                                          • 27. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                                            fredzarf Community Member

                                                            Hi from France.

                                                             

                                                            August, 2014, always the same problem, still not corrected on After Effects CC2014 on Mac OS X.

                                                             

                                                            When I apply a warp stabilizer on a (long) 4K shot, the project becomes much longer to load and save... even with 32 GB of RAM and 1 GB/s SSDs for data and cache.

                                                             

                                                            My project contains 15 long 4K rushes from BlackMagic Camera and more than 10 compositions (generated from xml files with "heavy" structures, more than 50 layers in each of them).

                                                             

                                                            Without stabilizer :

                                                            project size : 4 043 186 Bytes

                                                            opening project : 1 second

                                                            saving project : less than 1 second (unable to read text on the window)

                                                             

                                                            With stabilizer on a single 4K 4 minutes shot :

                                                            project size : 64 692 939 Bytes

                                                            opening project : 13.33 seconds

                                                            saving project : 26.08 seconds

                                                             

                                                            I spent lot of time to realize that the stabilizer was the bad guy in this story ! It's clear that it is guilty (with its huge hidden data) ! Even if the project size could not be smaller (that's logical, the stabilizer data should be stored somewhere)... it couldn't take so much time to open and save a project ! The stabilizer data are 1024-bit encrypted ones ?

                                                            • 28. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                                              Rick Gerard CommunityMVP

                                                              There's a basic flaw in your workflow. Warp Stabilizer is designed to improve the stability camera movement. A wise workflow would be to set up a separate project just to stabilize shots, then render the shots to a DI (digital intermediate) using a suitable production codec, then use that DI in your production. All of the overhead and cacheing that Warp Stabilizer must save kills any kind of productivity you may have had in your project.

                                                               

                                                              I do the same with any footage that requires significant processing before compositing or adding motion graphics. You'll save a boatload of time in the long run.

                                                              • 29. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                                                fredzarf Community Member

                                                                Thank you Rick, it's exactly what I've done when I discovered this bug in august, (and before that, I have generated a full-depth-resolution-quality-proxy to put the stabilizer out from my pipeline... but it's amazing that, even if the effect is turned off, its huge data has to be load and save each time !). So I put the warp stabilizer on a single comp separated from my work last week... I often do that in these cases, like you... but it's still annoying to see that we can't have everything in ONE great project !

                                                                • 30. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                                                  Rick Gerard CommunityMVP

                                                                  Everything in one big project is a horrible way to work on a project that has many parts. I've been making movies professionally for 40 years and it has never been a good idea. It goes back to clip bins and guillotine splicers and maximum 1000' reels on a moviola (that's about 10 minutes). No big production ever works this way. I haven't even produced any commercials that involved more than a couple of shots in a single big AEP file. It's just too hard to make the inevitable changes.

                                                                  • 31. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                                                    fredzarf Community Member

                                                                    Like you (and every pros) I make a project for each shot of a film (not for the entire movie !)... but I work (and teach) in visual effects since 20 years and sometimes I really need to get hundred of reels on the same project for huge compositing. I try to use less proxies, I try to work natively in linear 32-bit color space projects, so I can keep highlights from RAW sources to keep the best quality when I apply strong color grade (or apply FX like smoke coming in front of a "burned" object, or when I add reflective 3D objects that needs HDR animated texture sources), I need power and stability, it's why I use Mac OS X, SSDs for data and cache, dual-aggregated ethernet links for NAS and thunderbolt links for external drives, 32 GB - at less - on each computer and CUDA cards everywhere. I use massive expressions (that slowdown After Effects)... but nothing is worse than the Warp Stabilizer effect when it's time to save ! And I have just experienced that problem last week (the previous shots I have to stabilize with the warp stabilizer these last two years were very short, not 4 minutes shot in 4K !). I think Adobe should correct that in a near future.

                                                                    • 32. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                                                      adobefishman Community Member

                                                                      It's January 2015 and thanks to your post I know I'm not going crazy.

                                                                      As soon as I pre-rendered my clips and deleted the effect my normal save times returned...instantly.

                                                                       

                                                                      Yes, doing a Warp stabilize in a separate project is a good workaround.

                                                                      But it's been MONTHS Adobe, since this came to light.

                                                                      Don't pretend this is a normal procedure. Time to fix what clearly is a bug.

                                                                       

                                                                      And expecting people to figure that out looks bad and causes frustration to users like us.

                                                                       

                                                                       

                                                                      Model Name: Mac Pro

                                                                        Model Identifier: MacPro5,1

                                                                        Processor Name: 6-Core Intel Xeon

                                                                        Processor Speed: 2.66 GHz

                                                                        Number of Processors: 2

                                                                        Total Number of Cores: 12

                                                                        L2 Cache (per Core): 256 KB

                                                                        L3 Cache (per Processor): 12 MB

                                                                        Memory: 16 GB

                                                                      • 33. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                                                        Rick Gerard CommunityMVP

                                                                        In my humble opinion this isn't really a bug, it's a very efficient way to store the warp stabilizing data. Inefficient does not equal bug, it just equals poor design.

                                                                         

                                                                        Warp Stabilizer is still an amazing tool, flawed as it is, and it's much better than anything we had included in After Effects before. I'm going to give them a break on the coding inefficiencies for now because I know from experience how incredibly difficult it is to come up with efficient ways to store and retrieve huge amounts of data. There are other more pressing issiues in my every day, let's make a living workflow that I would like to see resolved first.

                                                                        • 34. Re: Saving Projects with Warp Stabilized Footage takes Forever
                                                                          adobefishman Community Member

                                                                          I appreciate the latitude you are giving the programers. They have a lot on their plate, to be sure.

                                                                          That fact that it isn't commonly understood leads me to believe it is something Adobe is either not aware of, or simply doesn't care. If the former then I'd say, based on my Adobe subscription and the frequency of updates updates, January is enough time to throw out a minor version, upgrade solution.

                                                                          I won't assume the latter.

                                                                          But it seems unfair not to warn the user that something as basic as "saving" will be severely compromised. If inefficient code is to blame, a small "don't ask me again" warning box would be a nice way to to users to be mindful of what the operation will do. Even that would have been a reasonable thing to include by now.

                                                                          And I'm not aware of any other function application in Adobe that causes such a slow down.

                                                                          But perhaps most importantly is the fact that user may blame the software in general for a problem they don't understand. I, for one, kept plodding through my saves assuming it was just sluggish software. Over time people can start to feel that Ae isn't a very robust platform when in actuality the reason is quite benign and has a workaround.

                                                                          It is worth noting that I use an Avid Media Composer to edit and it's warp stabilization has been baked in for years now (8yrs?). At no time has it compromised my saving of projects or bins with unnecessarily long wait times. And it's stabilization is both faster and better results than Ae (albeit not quite as flexible to modify).

                                                                          To sum it up, I'm not here to beat up Adobe. But my experience tells me the software issue that is here seems rectifiable... and overdue.

                                                                           

                                                                          ><({(º>