22 Replies Latest reply on Feb 10, 2009 10:24 PM by Harbs.

    [CS4 JS] Scripting Indexes

    frameexpert Level 4
      My client is trying to generate an index for a book with thousands of index entries. It takes about 4 hours and then gives an error message about bad characters, or something like that, and then it dies. (Apparently, this is a known issue.)

      One solution we are discussing is to create a JS script to collect the index data and create an index story, thus bypassing the built-in generation. To begin experimenting, I tried this:

      doc = app.activeDocument;
      
      index = doc.indexes[0];
      for (var i=0; i < index.allTopics.length; i++) {

        $.writeln(i+' '+index.allTopics[i].name);

      }


      The document has about 1,700 index entries in it. When I run this, it takes about 5 seconds for each item to be written to the JS Console. At this rate, it would take over 2 hours to loop through all of the index topics! It seems to me that it shouldn't take that long just to loop through the topics.

      Does any one know if there is a problem with the index architecture? Thank you very much.

      Rick Quatro
      585-659-8267
        • 1. Re: [CS4 JS] Scripting Indexes
          Peter Kahrel Adobe Community Professional & MVP
          At each iteration the script asks InDesign to create an array of topics, so maybe you could speed things up a bit by going about it like this:
          topics = app.activeDocument.indexes[0].allTopics;
          
          topics_length = topics.length;
          for (var i=0; i < topics_length; i++) {
            $.writeln(i+' '+topics[i].name);
          }


          As there are so many topics, this could make a difference.

          Peter
          • 2. Re: [CS4 JS] Scripting Indexes
            Level 1
            Hi Rick,

            Two things:

            1. Large arrays of text references are very slow in ExtendScript.

            2. Try running the script via app.doScript and turn off Undo. Often, the performance problem is related to InDesign's Undo feature (which writes to disk with each change to the document database).

            Thanks,

            Ole
            • 3. Re: [CS4 JS] Scripting Indexes
              frameexpert Level 4
              Hi Peter,

              I am a bit embarrassed, but you are exactly right. When I get the topics ahead of time, it cruises right through the list! Thank you very much.

              Ole, thanks for the suggestion. I will try this later in the process.

              Rick
              • 4. Re: [CS4 JS] Scripting Indexes
                Peter Kahrel Adobe Community Professional & MVP
                Ole,

                Which of the undo enumerations switches off Undo? None of the names Of the four listed in the OMV (fastEntireScript, entireScript, scriptRequest, autoUndo) suggest that they switch off undo.

                Peter
                • 5. Re: [CS4 JS] Scripting Indexes
                  Level 1
                  Hi Peter,

                  Sorry, it's not really quite the same as turning it completely off. Try fastEntireScript or entireScript--these mean you get one undo step, rather than thousands.

                  Thanks,

                  Ole
                  • 6. Re: [CS4 JS] Scripting Indexes
                    frameexpert Level 4
                    Ole,

                    Is it only possible to use doScript when calling an external script? Or, let me phrase it differently: can I set the undoMode for the current script without calling it from another script? Thanks.

                    Rick
                    • 7. Re: [CS4 JS] Scripting Indexes
                      Level 1
                      Olav,

                      is it really true, that reading something - without changing anything in document - will affect Undo ?

                      robin
                      --
                      www.adobescripts.com
                      • 8. Re: [CS4 JS] Scripting Indexes
                        Peter Kahrel Adobe Community Professional & MVP
                        Thanks, Ole.
                        • 9. Re: [CS4 JS] Scripting Indexes
                          Harbs. Level 6
                          Ole,

                          Are you sure that collapsing the undo stack really increases performance?

                          My understanding is that the undos are still recorded the same way, but
                          they are just collapsed into one general item in the undo stack. If
                          anything, there should be more overhead (collapsing them) -- not less.

                          I guess I should really do performance tests to find out...

                          --
                          Harbs
                          http://www.in-tools.com
                          • 10. Re: [CS4 JS] Scripting Indexes
                            Peter Kahrel Adobe Community Professional & MVP
                            Ole was right: doScript() with UndoModes.fastEntireScript gives the best results. The test was a document without any formatting, no undo history, containing 36,000 characters. The script creates a character style and sets underline to true. Then it finds all "A" and "a" (2,680 instances), changes the contents of each found instance with "x" and applies the character style.

                            Results:

                            UndoModes.fastEntireScript 22.6, 22.8, 23.1
                            UndoModes.autoUndo 25.8, 25.8, 25.9
                            UndoModes.entireScript 26.3, 26.4, 26.5
                            UndoModes.scriptRequest 29.9, 29,3, 29.2

                            Conclusion: fastEntireScript lives up to its name and wins hands-down.

                            My next step, I thought, was to add from now on this line at the beginning of labour-intensive scripts:
                            >app.activeScriptUndoMode = UndoModes.fastEntireScript;

                            But unfortunately, activeScriptUndoMode is read only. Time for a feature request.

                            Peter
                            • 11. Re: [CS4 JS] Scripting Indexes
                              Peter Kahrel Adobe Community Professional & MVP
                              The timings in post #10 are from a script run from the ESTK. It has been mentioned that running scripts from the palette is quicker, which is confirmed:

                              UndoModes.fastEntireScript 21, 21
                              UndoModes.autoUndo 50, 43 / 30, 24
                              UndoModes.entireScript 52, 33 / 31, 26
                              UndoModes.scriptRequest 24, 25

                              The timings for autoUndo and entireScript look bizarre here, I've no idea what was going on. Timings after the slash were obtained after restarting InDesign. Didn't improve. The downward trend of these two modes in seuccessive runs is strange.

                              Anyway, all this confirms that the quickest way to run a script is to have it fired up by doScript() using UndoModes.fastEntireScript.

                              Peter
                              • 12. Re: [CS4 JS] Scripting Indexes
                                Harbs. Level 6
                                The longer times for entireScript and autoUndo seems to confirm my
                                suspicion that there's extra overhead with collapsing the undo stack. I
                                wonder what's done differently with fastEntireScript, and why do we need
                                the regular entireScript if we are presented with a faster option?

                                The fact that the UndoModes are read-only is not really that big of a
                                deal. All you need to do is add to the beginning of the script:
                                var undoName = "Script Name";
                                app.doScript(main, undefined , undefined,UndoModes.fastEntireScript ,
                                undoName );

                                instead of:
                                main().

                                --
                                Harbs
                                http://www.in-tools.com
                                • 13. Re: [CS4 JS] Scripting Indexes
                                  frameexpert Level 4
                                  Back to an index question. I need to collect all of the index entries in an InDesign book and create an index. Can anyone suggest a good data structure that I can use to collect the index entries? I will need to collect the entries and the corresponding page numbers. After they are collected, I will need to do a primary sort on the entries and a secondary sort on the page numbers. Any suggestions would be appreciated.

                                  Rick Quatro
                                  585-659-8267
                                  • 14. Re: [CS4 JS] Scripting Indexes
                                    Peter Kahrel Adobe Community Professional & MVP
                                    Harbs,

                                    That looks promising, but when I try that, I get an error "UndoModes is undefined". Any ideas?

                                    Thanks,

                                    Peter
                                    • 15. Re: [CS4 JS] Scripting Indexes
                                      Peter Kahrel Adobe Community Professional & MVP
                                      Rick,

                                      Are you trying to mimick "Generate index" from the Index panel?

                                      Peter
                                      • 16. Re: [CS4 JS] Scripting Indexes
                                        Harbs. Level 6
                                        I'm telling you: somethings fishy with the enums in CS4. I've had this
                                        kind of problem on many occasions... :(

                                        try:

                                        app.doScript(main,undefined,undefined,1699964501,undoName);


                                        --
                                        Harbs
                                        http://www.in-tools.com
                                        • 17. Re: [CS4 JS] Scripting Indexes
                                          frameexpert Level 4
                                          Peter,

                                          Yes, I am trying to mimic "Generate Index". My client has a book with a large number of index entries. The index is taking about 4 hours to generate, and then it dies with an error message about an invalid character. I don't have the exact error message right now, but apparently this is a known issue with large indexes.

                                          So my goal is to roll my own index generator. Right now I am using an array of objects for the index entries. Each object has two members: entry, which is the index entry itself; and pages, which is an array of page numbers. Once I collect all of the entries in the book, sort them, and sort and remove duplicates from the page numbers, I should be able to write the index.

                                          Rick
                                          • 18. Re: [CS4 JS] Scripting Indexes
                                            Peter Kahrel Adobe Community Professional & MVP
                                            Harbs,

                                            If only I had remembered that UndoModes were introduced in CS4... Tried it in there and it works fine. Thanks very much for the pointer.

                                            Peter
                                            • 19. Re: [CS4 JS] Scripting Indexes
                                              Peter Kahrel Adobe Community Professional & MVP
                                              Rick,

                                              Here's (a variant of) a script that I sometimes use to deal with indexes. Its output is an array of strings of topics followed by their page numbers. Topics and numbers are separated by // so you can find them easily. Subtopics are separated by #. So you get something like this:

                                              Basque//1, 2, 3, 4
                                              Romance#Italian//1, 2, 3, 4

                                              Peter

                                              #target indesign;
                                              

                                              rough_index = manual_index (app.activeDocument);

                                              function manual_index (doc)
                                                 {
                                                 var indexx = [];
                                                 t = app.activeDocument.indexes[0].allTopics;
                                                 for (i = 0; i < t.length; i++)
                                                    {
                                                    prefs = t[i].pageReferences.everyItem().getElements();
                                                    if (prefs.length > 0)
                                                       {
                                                       topic = topic_path (t[i], t[i].name);
                                                       pages = find_pages (prefs);
                                                       indexx.push (topic + '//' + pages)
                                                       }
                                                    }
                                                 return indexx
                                                 }

                                              function topic_path (top, str)
                                                 {
                                                 if (top.parent.constructor.name == 'Index')
                                                    return str;
                                                 else
                                                    return topic_path (top.parent, top.parent.name + '#' + str)
                                                 }

                                              function find_pages (page_refs)
                                                 {
                                                 var temp = [];
                                                 for (var i = 0; i < page_refs.length; i++)
                                                    {
                                                    try {temp.push (page_refs[i].sourceText.parentTextFrames[0].parent.name)}
                                                    catch (_){}
                                                    }
                                                 return temp.join (', ')
                                                 }
                                              • 20. Re: [CS4 JS] Scripting Indexes
                                                frameexpert Level 4
                                                Hi Peter,

                                                Thanks. I will have to go through your code and digest it. I have mine working at the book level, except for sorting the entries and writing the index to InDesign. Right now, it writes the index to a text file just so I can test the results, performance, etc.

                                                Thank you very much for your help.

                                                Rick
                                                • 21. Re: [CS4 JS] Scripting Indexes
                                                  (Simon_Paterson) Level 1
                                                  Harbs, sometimes you need to do more investigation before you post...
                                                  • 22. Re: [CS4 JS] Scripting Indexes
                                                    Harbs. Level 6
                                                    Simon_Paterson@adobeforums.com wrote:
                                                    > Harbs, sometimes you need to do more investigation before you post...
                                                    >


                                                    Huh?

                                                    I don't remember the details right now but I've definitely had
                                                    situations (soon after I started working in CS4) where the enums did not
                                                    work (I got an error "xxx in undefined" -- xxx being the enumeration
                                                    object) but the numerical equivalent did.

                                                    I wasn't able to work out what caused the error. Any time I got such an
                                                    error, I simply switched to the numerical equivalent and went on working...

                                                    --
                                                    Harbs
                                                    http://www.in-tools.com