11 Replies Latest reply on Jun 8, 2014 9:20 AM by Trevorׅ

    How to cleam memory in InDesign.

    cero

      Dear all,

       

      We are developing a extension for Adobe InDesign and after many months of development we are still stuck with various things we cannot find a proper solution to them. The extension exports a lot of images, texts, creates new pages, basically we are exporting all the content in a document as long as 300-400 pages in different assets, in JPG and PNG formats. So is really memory consuming

       

       

      When running the extension we have developed, the main thread is bloqued and the InDesign is not responding and if the export we are doing is really large it just crashes InDesign. We would like to reléase memory after each page is finished, but we cannot do so. Once the complete export is finished, then memory is released, but not while the export is happening

       

      As we export the memory is increasing more and more without releasing anything, we have tried different options, but seems like we never get to release memory as everything is blocked

       

      How can we release memory while the extension is running ? Or is there a way to stop the extension each iteration so it releases memory?

       

       

       

       

      We use the extension manager and flash builder.

       

      Thanks!

      Cesar.

        • 1. Re: How to cleam memory in InDesign.
          mim_Armand Level 1

          Hi,

          I assume your code is in JavaScript, then here is my thoughts, (I'm pretty new in writing for in-design as well but it may help!):

           

          • about the blocking thread, I think it is because you used a 'for' or 'while' cycle and since JS is a single process/thread language it will pretty much block that thread till it's done, you may want to use an unblocking technique or give back the thread to OS every once in a while (like each 10 page for example) and take it back again (which may be a good time to call GC too).
          • for memory management, JavaScript has its own Garbage collector which will run when it thinks it's needed, however, if you keep live references to memory it will never get released, although in pure JavaScript there is no way to manage garbage collector (or call it when you need to), but good news is that apparently it is available in Extend-script through the function $.gc()! , bad news is that it probably wont help you (since I guess your problem is coming from somewhere else - IE not recycling memory through your cycles or simply blocking it) there is techniques to work it around as well: like Object pooling for example which is a very efficient way to manage memory in JavaScript and in programming in general which I guess will solve your memory problem entirely (instead of keeping hundreds of pages in memory you can keep just one all the time and replace it with the next page when you were done with the first one!)

           

          hope it helps, good-luck, mim

          • 2. Re: How to cleam memory in InDesign.
            Jump_Over Level 5

            Hi,

             

            However deep of memory consuming is your code there is a "undo" stack created by ID as well on the fly.

            So saving a doc or using other way to cut this stack down - could be a defender against memories error.

             

            Jarek

            • 3. Re: How to cleam memory in InDesign.
              cero Level 1

              Thanks Armand

               

              How can we unblock the thread and back to OS every once in a while ? And how we can take it back again ? Our main issue is that we don't know how to unblock the thread as gets totally locked once we start the process and hence no way to free memory and InDesign ends up crashing all the time the document is large.

               

              Now we are calling GC every iteration, and set null all objects to reléase.

              Any ideas?

               

              Thanks so much.

              Cesar.

              • 4. Re: Re: How to cleam memory in InDesign.
                mim_Armand Level 1

                You welcome,

                It really depends on your code Cero, but for example if you have a for or while loop that goes for hundreds of cycles you can break it down to smaller portions, for and loop block a thread during their execution, that mean each while/for loop is a single process request no matter how big or intense it'd be, it should be done before that CPU core/thread accepts another request, but if you break that to two cycles, thread will be freed by the first request and taken back by the second one (therebetween cpu can decide about priorities and runs some short, high priority tasks before getting to work on the second loop). again, it depends on your code and your needs but for example to break a 100 time loop into 10 x 10 ones you can do:

                 

                var arrayLength = 100;
                
                /*Before:*/
                for (var i = arrayLength - 1; i >= 0; i--) {
                    exportThePage(i);
                };
                
                /*After:*/
                var breakPoint = 10;
                var neededCycles = arrayLength / breakPoint; //obviously you need to improve these aspects since it breaks if you have 103 cycles for example!
                var currentCycle = 0;
                function loopBreaker(){
                    if(breakPoint < arrayLength && currentCycle <= neededCycles){
                        currentCycle++;
                        mainLoop(currentCycle);
                    } else if (breakPoint >= arrayLength) {
                        alert('no need for me! just call a simple loop!')
                    } else {
                        alert('Im done here!');
                    }
                
                }
                function mainLoop(currentCycle){
                    for (var i = breakPoint - 1; i >= 0; i--) {
                        exportThePage((breakPoint * (currentCycle - 1)) + i);
                        alert('just finished workin on page: '+(breakPoint * (currentCycle - 1)) + i);
                    }
                    loopBreaker();
                }
                
                
                

                 

                Now I didn't have a chance to test this at all (I may be able to test it tomorrow at university) (Edit: unsurprisingly there were a little error which I fixed), in adition it makes it harder to do such a thing in extendScript in absence of javascript timing events but I guess it will work if your page export process is heavy enough to prevent this code to through an stack overflow error (otherwise you need to increase breakPoint variable, or use a short sleep period or you simply don't need it really), I also hope to not have a syntax error here so please re-check it yourself.

                 

                About using GC it may (or may not) solve your memory problem but most certainly wont help any on your cpu load, it's better to re-use objects and not to destroy them every time.

                 

                PS: ah I forgot to say but there is no need really to mention that you need to change and adapt above code cause it will break easily if you have 103 pages for example instead of 100, I just wanted to show one of the approaches I'd think of,

                 

                hope it helps and gl ,

                m,

                • 5. Re: How to cleam memory in InDesign.
                  Laubender Adobe Community Professional & MVP

                  @mim – your approach is promising. I already had implemented something like that – breaking a big loop to "some" portions – reducings the overall time it takes. However, I was working on big tables and their contents. Additionally I saved the InDesign document with a Save As every time a portion of the loop ended.

                   

                  That all helped a bit. I found, that time for executing the script dropped to 20%. And it was more stable.

                  Of course there is a point where reducing the individual loop portions (plus saving) would increase the time of execution. Saving needs its time, too.

                   

                  But there are other factors as well: sometimes redraw issues.

                   

                  Not your issue here, but:

                  By working with tables I found that converting text to a new table was considerably faster than changing the contents of an existing table.

                   

                  Usually you would write the contents of a table to an array, changing the contents there. Rearranging columns, sorting etc. Then writing back that contents to the table. That was rather slow. I was looking for an alternative method and found: Writing the contents of the changed array to a text file, removing the old table, imported the text file as text and converting that to a table was tremendously faster than writing back the contents of the array to the existing table.

                   

                  Uwe

                  • 6. Re: How to cleam memory in InDesign.
                    mim_Armand Level 1

                    Thx Laubender, I'm glad that it helped a bit, (tho I'm confused but I guess you guys work together)

                    that's good to know as well your solution to edit big tables, very good approach on that,

                     

                    Goodluck with the rest of your project,

                    mim,

                    ====================================================

                    PS: may you want to mark the thread as answered if you consider it so,

                    • 7. Re: Re: How to cleam memory in InDesign.
                      Trevorׅ Adobe Community Professional

                      Hi Cesar,

                       

                      I'm also making an export script, (not extension), mine seems to be quite different to yours but we'll might see when both are published.

                       

                      Off the point.

                      The first thing you have to be aware of is using flash builder and extension manager, If you are making a flash based extension, from what I understand it's not going to work for the next version of InDesign which is switching to HTML5 based panels and dropping support for flash based panels (you'll still need the flash based panels to support the current version (9) and below. This is going to be a major pain for all extensions for the next lot of years that all of them will need 2 completely different interface bases to be compatible with CC(1), CS6 (which is going to be around for a long time for obvious reasons) and below on the one side and CC(2) + on the other.  I can't actually blame Adobe for this one as there should be some serious benefits in the HTML5 panels.

                       

                      Back to the point.

                      See Ariel's Phenomenal optimization technique! which is based on the same Idea of breaking down the script into small parts.

                      See also Re: Big performance issue while removing tabs by indents including the link brought by Xavier in 7 over there. Loic reduced his script run time from 6 hours + to 90 seconds by using efficient techniques.

                      I made a script for making swatch books (sort of thing than Pantone publish - involved a quite a bit of maths and color conversions) it was workable for 3,000 - 4,000 swatches but when it got to 15,000 or so swatches the machines wouldn't budge one bit.  Separating the output into separate document  (which any I needed to do because the proofing and print machines can only handle up to 27 spot colors) resulted in the script working very efficiently.

                       

                      As Uwe mentioned redraw can make a big difference and even more so hidden documents app.open(myFile, false)

                       

                      Using myObjectCollection.everyItem().getElements().slice(0). Very significant for big collections.

                       

                      Better still don't make big collections in the first place.  Don't if possible try process all the export items in the document myItems = doc.allGraphics or myItems = doc.spreads.everyItem().allGraphics rather make a collection myItems = doc.spread[n].allGraphics this will take up much less memory and work much quicker.

                       

                      Try playing with $.memCache see if increasing or reducing the memory allocation makes a difference.  Please let me know if it does!

                       

                      There's a very lot of methods that would depend on the actual script the differences can be huge, from scripts that just won't move to taking less than 2 minutes.

                       

                      Don't forget to report back with the results

                       

                      Trevor

                      • 8. Re: How to cleam memory in InDesign.
                        cero Level 1

                        @mim_Armand : Your method "for cycles" works perfect!! Solves our memory problem !! We reimplement our loop method for your version, we break the loop every 5 iterations and the memory has dropped.

                         

                        @Laubender: We try saving document in a "big loop" but has not help us with this issue

                         

                        @Trevorׅ: We are programming our extension with Actionscript/Flex. It's not going to work for the next versions of InDesign ? Are those official news?

                         

                        We changed all our big collections and works fine, is a good optimization!!

                        Now we resolve a Little our memory problem, but doing tests we try to put a Dialog (Modal Dialog). This Dialog stops all processes and frees a lot of memory, even some extra comparet to Armand suggestion, the problem is that this window needs to be clicked by the user in order to work. do you know how can web destroy/close the Dialog automatically? Like a timer for example, the message appears and if you don't hit ok it just fades. If it's possible the memory will work perfectly!! Doesn't work with Window or panel, only works with Modal Dialog...

                         

                        Thanks a lot for the inputs, this is amazing help!

                        Cesar.

                        • 9. Re: How to cleam memory in InDesign.
                          mim_Armand Level 1

                          @cero, Hey, I'm really glad that it helped, I have the answer to your new question (I mean I guess!) but you should not do that in the same topic, if you got the answer to your current question (How to clean the memory in indesign) you should mark the thread as answered and start a new topic for your new question, it will help to keep the forums clean and make it easier for other people to find the answer if they happened to have the same question, Here, read the "Forum Does and Don'ts".

                          But as a quick answer, you should not do that really! Modal Dialogue has it's own usages, it's not intended to be used in the way you described, you should use it just to inform the user about something or when you need a user interaction etc. . in the other hand there is no timing event in extend-script (no timeout, no setinterval) so you can't use  timing functions in a non-blocking fashion, but since the date object exist you can create a blocking set-interval function yourself which may trigger a function (to close the dialogue) after desired milliseconds, you should look at the documentation to see what's available to hide() or destroy() (??!) a dialogue ... but again, it's not a good idea anyway!

                           

                          Goodluck,

                          mim,

                           

                          (ps: you can mention me in your new thread so I'll get notified (somedays I can't check the forums) and I'll do my best to help if any)

                          • 10. Re: How to cleam memory in InDesign.
                            cero Level 1

                            Thanks for all your help Mim!

                            • 11. Re: Re: How to cleam memory in InDesign.
                              Trevorׅ Adobe Community Professional

                              Cesar

                               

                              See Introducing HTML5 extensions | Adobe Developer Connection

                               

                              Regarding self-closing alerts, on Mac it's easy

                              app.doScript ("""tell application "Finder"
                                  display dialog "Hello, World!" giving up after 2
                              end tell
                              """, ScriptLanguage.APPLESCRIPT_LANGUAGE)
                              

                              but on widows it looks like the only way to do it is making a vbs file as the doScript doesn't work in this case.  I doubt that will solve your problem.

                              See my post Re: Export and attach it into a mail in 'Outlook'? from line 47 for an example

                               

                              HTH

                               

                              Trevor