32 Replies Latest reply on Feb 17, 2017 4:12 AM by PeteBaumgartner

    Finding Unassigned Glyphs

    PeteBaumgartner Level 1

      If we import or enter a text code (e.g. a Katakana character )  for which a font has no Glyph, then Indesign will display an x'd rectangle.

       

      Is it possible to find/scan for these with a script?

       

      Thanks !

        • 2. Re: Finding Unassigned Glyphs
          absqua Level 4

          Depending on what you're trying to do—i.e., is a simple pass/fail enough, or do you need to programmatically do something to each missing glyph—the preflight might work.

           

          Jeff

          1 person found this helpful
          • 3. Re: Finding Unassigned Glyphs
            Laubender Adobe Community Professional & MVP

            @Pete – just a thought:

             

            if we are looking for all none-white space characters and try to convert  character by character to outlines, and this convertion will fail, would that be the proof that we found a unassigned glyph?

             

            Here an example. Characters that cannot converted to outlines will be formatted with the character style "NotDefinedGlyph", that you have to prepare before running this snippet:

             

            //Very slow. Don't try this on large stories:
            var d=app.documents[0];
            
            var sChars=d.stories[0].texts[0].characters;
            var sCharsLength = sChars.length;
            
            //Not foolproof!
            //False positives with ligatures like "fi" or "ffi":
            for(var n=0;n<sCharsLength;n++){
                
                //Do not create outlines on white space characters:
                if(sChars[n].contents.match(/\S/)){
                    try{
                        var myOutlineArray = sChars[n].createOutlines(false);
                        try{
                        myOutlineArray[0].remove();
                            }catch(e){};
                        }catch(e){
                            //Let's mark it with a character style in case it cannot be converted to outlines:
                            sChars[n].appliedCharacterStyle = "NotDefinedGlyph";
                            $.writeln(sChars[n].contents);
                            };
                    };
                
                };
            

             

            Downside: this code would run very, very slow…
            And: it's NOT foolproof. There are false positives with ligatures like "fi" or "ffi" etc.pp.

             

            Uwe

            1 person found this helpful
            • 4. Re: Finding Unassigned Glyphs
              Peter Kahrel Adobe Community Professional & MVP

              Everybody please head to the feature request page: http://www.adobe.com/support/feature.html

              I've been asking for script support for this for years, but my feeble voice hasn't had much success.

              The more requests, the bigger the chance we get the feature.

               

              Peter

              1 person found this helpful
              • 5. Re: Finding Unassigned Glyphs
                PeteBaumgartner Level 1

                All;

                 

                Thanks for the ideas. I will continue to tinker.

                 

                I did add an entry to the Feature requests.

                 

                pjb

                • 6. Re: Finding Unassigned Glyphs
                  Pickory Level 3

                  Hello Peter,

                   

                  Spec out what you would like to do. I am more than happy to try and write a 'FREE' plugin that your scripts can call.

                   

                  All the best.

                   

                  P.

                  • 7. Re: Finding Unassigned Glyphs
                    Peter Kahrel Adobe Community Professional & MVP

                    Easy to spec out: a method that works at document level and lists glyphs/characters used in the document which are not present in the used fonts. For example, myDocument.missingGlyphs() would return an object that in some way tells you what glyphs are missing from which font. Something along these lines:

                     

                    missing = app.activeDocument.missingGlyphs();

                    for (m in missing)

                       $.writeln (m, missing[m])

                     

                    So the keys would be the font names (Times-Bold, Arial-Italic) and the values, the missing glyphs in those fonts. But the data model doesn't matter, as long as you get a list of missing glyphs per font.

                     

                    Would be great if you could write such a plug-in.

                     

                    Peter

                    • 8. Re: Finding Unassigned Glyphs
                      Pickory Level 3

                      Hello Peter,

                       

                      It would seem to me that missing glyphs are reported as the text is composed. We could cause the document to compose by faking a print / redraw. I could then capture missing glyphs in the print / redraw process and then report the list back to your scripts.

                       

                      Quite an overhead and a bodge.

                       

                      I will give this some more thought.

                       

                      All the best.

                       

                      P.

                      • 9. Re: Finding Unassigned Glyphs
                        [Jongware] Most Valuable Participant

                        For CS4 (and hopefully newer): create a preflight profile that checks just the bad'uns, then flesh out the relevant details. The scripting engineers went a little bit crazy on making this scriptable, it seems.

                         

                        (This code only returns the results after 2 minutes, and it only shows them. So please none of that "plz can u send ur complete script" crap if you plz.)

                         

                        if (!app.preflightProfiles.item("Check glyphs only").isValid)
                        {
                                  glyphsOnlyProfile = app.preflightProfiles[0].duplicate();
                                  glyphsOnlyProfile.name = "Check glyphs only";
                                  for (i=0; i<glyphsOnlyProfile.preflightProfileRules.length; i++)
                                            glyphsOnlyProfile.preflightProfileRules[i].flag = PreflightRuleFlag.RULE_IS_DISABLED;
                                  glyphsOnlyProfile.preflightProfileRules.itemByName("ADBE_MissingGlyph").flag = PreflightRuleFlag.RETURN_AS_ERROR;
                        }
                        
                        
                        glyphsOnlyProfile = app.preflightProfiles.itemByName("Check glyphs only");
                        
                        
                        activeProcess = app.preflightProcesses.add (app.activeDocument, glyphsOnlyProfile);
                        activeProcess.waitForProcess(120);
                        
                        
                        result = {};
                        for (i=2; i<activeProcess.aggregatedResults[2].length; i++)
                                  if (result[activeProcess.aggregatedResults[2][i][1]])
                                            result[activeProcess.aggregatedResults[2][i][1]]++;
                                  else
                                            result[activeProcess.aggregatedResults[2][i][1]] = 1;
                        alert (result.toSource());
                        
                        • 10. Re: Finding Unassigned Glyphs
                          [Jongware] Most Valuable Participant

                          Gosh -- trying it on a hundred pages of text where I replaced all of the body font with Symbol made ID crash

                           

                          Thread 0 Crashed:: Dispatch queue: com.apple.main-thread

                          0   ???                                     0xac8f4630 _XHNDL_trapback_instruction + 0

                          1   com.adobe.InDesign.Package and Preflight          0x0fd477ec GetPlugIn + 554380

                           

                          Oh well, better use it "within reasonable limites" then.

                          • 11. Re: Finding Unassigned Glyphs
                            PeteBaumgartner Level 1

                            I did finally manage to script a list of all the unassinged glyphs in my document - with .findGlyph().

                             

                            This sample just writes to the extendscript toolkit js console.

                             

                            [CS6]

                            -------------------------------------------------------------------------------

                            var myDocument=app.activeDocument;

                            var allFound = [];

                             

                            var numFonts = myDocument.fonts.length;

                             

                            for (var z=0;z<numFonts;z++) {

                                allFound = [];

                                app.findGlyphPreferences = NothingEnum.nothing;

                                app.findGlyphPreferences = NothingEnum.nothing;

                             

                                app.findGlyphPreferences.glyphID  = 0;

                                app.findGlyphPreferences.appliedFont = myDocument.fonts[z].name;

                             

                                allFound = app.findGlyph ();

                                if (allFound.length>0) {

                                    $.writeln(allFound.length+ ' Unassigned Glyphs in ' + myDocument.fonts[z].name );

                               

                                    for (var y=0;y<allFound.length;y++) {

                                         $.writeln( allFound[y].contents+ '   ' +allFound[y].contents.charCodeAt(0) );

                                    }

                               }

                            }

                               

                            $.writeln('--done--');

                            -------------------------------------------------------------------------------

                            • 12. Re: Finding Unassigned Glyphs
                              [Jongware] Most Valuable Participant

                              Wow, that is excellent!

                              • 13. Re: Finding Unassigned Glyphs
                                Pickory Level 3

                                That is very neat!

                                 

                                Thanks, that has let me off the plugin.

                                 

                                P.

                                • 14. Re: Finding Unassigned Glyphs
                                  Peter Kahrel Adobe Community Professional & MVP

                                  Brilliant! Thanks very much. I shall use this script a lot I'm sure.

                                   

                                  Peter

                                  • 15. Re: Finding Unassigned Glyphs
                                    Laubender Adobe Community Professional & MVP

                                    @Pete – Wow! That is working. Just tested in InDesign CS5.5,

                                    At least with my sample document I put up very fast.

                                     

                                    "app.findGlyphPreferences.glyphID = 0" seems to be the proof…

                                     

                                    Just one thing:

                                    Instead of "allFound = app.findGlyph();" I would change the scope to one document:
                                    "allFound = myDocument.findGlyph();"

                                     

                                    Thank you a lot!

                                     

                                    Uwe

                                    • 16. Re: Finding Unassigned Glyphs
                                      Peter Kahrel Adobe Community Professional & MVP

                                      Pete,

                                       

                                      It might be useful to display the hex/unicode value of missing characters, like this:

                                       

                                      . . . allFound[y].contents.charCodeAt(0).toString(16)

                                       

                                      Peter

                                      • 17. Re: Finding Unassigned Glyphs
                                        Laubender Adobe Community Professional & MVP

                                        @Pete – another thing: don't know why, but with that method you cannot find unassigned glyphs in overset texts.

                                        Wheras  in the UI that is possible…

                                         

                                        Example inDesign CS5.5:

                                        if you are searching for "ID: GID/CID" value "0" in the UI Glyph search, for a given font you can find all the unassigned glyphs of that font even in overset text.


                                        With your script snippet this is not possible.

                                        Strange…

                                         

                                        Sorry. I was too fast in testing…
                                        Same in UI with glyph search: you cannot find glyphs with "ID: GID/CID" value "0" in overset text…

                                         

                                        Uwe

                                         

                                        Message was edited by: Laubender

                                         

                                        Message was edited by: Laubender

                                        • 18. Re: Finding Unassigned Glyphs
                                          Laubender Adobe Community Professional & MVP

                                          To expand this a bit: In the UI that is only possible, if you define two items in glyph search:

                                           

                                          "ID: GID/CID" of value "0" together with a specific glyph will find that glyph in overset text.

                                           

                                          Wheras "ID: GID/CID" alone will find all glyphs with value "0" in not overset text.

                                           

                                          All tests in InDesign CS5.5 v7.5.3.

                                           

                                          Uwe

                                          • 19. Re: Finding Unassigned Glyphs
                                            Trevorׅ Adobe Community Professional

                                            Very nice, works on CS5

                                             

                                            It would be useful to be able to find missing font styles as well.

                                             

                                            If one of you bright guys could add that to the script it would be apreaciated

                                             

                                            Trevor

                                            • 20. Re: Finding Unassigned Glyphs
                                              PeteBaumgartner Level 1

                                              Thanks. I plan to include this in a new extension we are wroking for text-checking files pre- and post- translation.

                                               

                                              - pjb

                                              • 21. Re: Finding Unassigned Glyphs
                                                Trevorׅ Adobe Community Professional

                                                In the meantime mark your answer as correct, helps with the forum search.

                                                 

                                                Good luck with the extension

                                                 

                                                Trevor

                                                • 22. Re: Finding Unassigned Glyphs
                                                  David W. Goodrich Level 3

                                                  This is really great.  I must say I appreciate Peter Kahrel's modification to yield hex if only because that is how I'm used to seeing character codes.  For obvious reasons, "toString(16)" messes up 2nd-plane Chinese (e.g. U+2A64A, available in MingLiU-ExtB), but on my system (IDCS4 under Win7) the console still shows the correct character. 

                                                   

                                                  On the other hand, there are CJK fonts with "extra" chars. that don't have a Unicode encoding but only a glyph ID.  For example, the char. in Adobe Ming Std. that ID's glyph palette reports as "CID: 17382" gets listed on the console as U+001a.  In my work, such cases are few: the same font contains the same glyph at U+8B4C, i.e., a code-point where standard input systems can find it; and the determined ID user who inputs the CID no. will find that ID turns the coding for the glyph into U+0020 when exporting to PDF -- maybe okay for print but not so useful as publishing goes electronic. 

                                                   

                                                  Japanese might be a little different: the Kozuka fonts bundled with ID contain quite a few "hors catégorie" items, and occasionally I have used the circled or squared two-digit numbers.  But these are things I add -- I never see them in the manuscripts I receive -- and I generally apply a character styles to all CJK protect them from accidental stripping of the font attribute.

                                                   

                                                  David

                                                  • 23. Re: Finding Unassigned Glyphs
                                                    [Jongware] Most Valuable Participant

                                                    David wrote:

                                                    > For obvious reasons, "toString(16)" messes up 2nd-plane Chinese (e.g. U+2A64A, available in MingLiU-ExtB), but on my system (IDCS4 under Win7) the console still shows the correct character.

                                                     

                                                    Just for added clarification: the "obvious" (grin) reason is that Plane 2 characters consist of not one Unicode, but TWO! And Pete's great discovery -- thinking well outside of the box! -- will probably report them, but it only sees one character at a time.

                                                     

                                                    InDesign correctly handles the two successive Unicode character, although it's weird you can move your cursor "between" the two and then accidentally mess up your text.

                                                     

                                                    For an example w/o being able to read Chinese, Check out the Apple Color Emoji font. You won't see the colors in InDesign, for unrelated reasons, but if you insert one of the icons into ID using the Glyphs palette, you can see it actually consists of two codes.

                                                    • 24. Re: Finding Unassigned Glyphs
                                                      Peter Kahrel Adobe Community Professional & MVP

                                                      Plane-1 and higher characters can be handled as follows. Replace this line in Pete's code:

                                                       

                                                      $.writeln( allFound[y].contents+ '   ' +allFound[y].contents.charCodeAt(0) );

                                                       

                                                      with this one:

                                                       

                                                      $.writeln( allFound[y].contents+ '   ' + get_hex (allFound[y]));

                                                       

                                                      and add this function to the script:

                                                       

                                                      function get_hex (ch)

                                                          {

                                                          function pad (n) {return ('0000' + n).slice(-4);}

                                                          try

                                                              {

                                                              if (ch.contents.length == 2)

                                                                  {  // plane-1 and higher algorithm algorithm adapted from

                                                                      // http://www.russellcottrell.com/greek/utilities/SurrogatePairCalculator.htm

                                                                  var H = parseInt (ch.contents.charCodeAt(0).toString(16), 16);

                                                                  var L = parseInt (ch.contents.charCodeAt(1).toString(16), 16);

                                                                  var s = ((H - 0xD800) * 0x400) + (L - 0xDC00) + 0x10000;

                                                                  return 'U+' + s.toString(16);

                                                                  }

                                                              // Plane-0 character; pad hex up to four characters

                                                              var s = ch.contents.charCodeAt(0).toString(16);

                                                              return 'U+' + pad (s);

                                                              }

                                                          catch (_) {return 'U+????'}

                                                          }

                                                       

                                                      Peter

                                                      • 25. Re: Finding Unassigned Glyphs
                                                        PeteBaumgartner Level 1

                                                        Peter.

                                                         

                                                        Thanks - works well !

                                                         

                                                        pjb

                                                         

                                                        e.g.,

                                                         

                                                        4 Unassigned Glyphs in Myriad ProRegular

                                                        十   21313  U+5341

                                                        分   20998  U+5206

                                                        な   12394  U+306a

                                                        利   21033  U+5229

                                                        --done--

                                                        • 26. Re: Finding Unassigned Glyphs
                                                          Peter Kahrel Adobe Community Professional & MVP

                                                          There's a bug that prevents the script from working with font styles other than Regular (and probably Roman). The problem is that doc.fonts[z].name returns e.g. Times ItalicItalic and Helvetica BoldBold. The workaround is to change this line:

                                                           

                                                          app.findGlyphPreferences.appliedFont = doc.fonts[z].name;

                                                           

                                                          with these two:

                                                           

                                                          app.findGlyphPreferences.appliedFont = doc.fonts[z].fontFamily;

                                                          app.findGlyphPreferences.fontStyle = doc.fonts[z].fontStyleName;

                                                           

                                                          Peter

                                                          • 27. Re: Finding Unassigned Glyphs
                                                            David W. Goodrich Level 3

                                                            I'd only tried "regular" font styles, and didn't have occasion to notice the regular/roman bug, so I really appreciate Peter K.'s going back and ferreting this out. For those who know even less about scripting than I do, I got his fix to work in the code presented earlier in this thread by adjusting the three "doc." references in his before and after lines to "myDocument."

                                                             

                                                            David

                                                            • 28. Re: Finding Unassigned Glyphs
                                                              PeteBaumgartner Level 1

                                                              Peter

                                                               

                                                              Thanks !

                                                               

                                                              Pete

                                                              • 29. Re: Finding Unassigned Glyphs
                                                                Peter Kahrel Adobe Community Professional & MVP

                                                                Pete,

                                                                This is the only way I can contact you. Hope you don't mind that I tarted up the script based on your excellent method with an interface and posted it on my site at http://www.kahrel.plus.com/indesign/missing_glyphs.jsx

                                                                 

                                                                Peter

                                                                1 person found this helpful
                                                                • 30. Re: Finding Unassigned Glyphs
                                                                  daitranthanhoa Level 1

                                                                        oIndesign.FindGlyphPreferences = 1851876449

                                                                          oIndesign.FindGlyphPreferences.glyphID = 0

                                                                          oIndesign.FindGlyphPreferences.AppliedFont = oDocument.TextFrames(1).Characters(13).appliedfont.name // 小塚明朝 Pro

                                                                           Dim oFind As Object = oDocument.FindGlyph()

                                                                  If set   oIndesign.FindGlyphPreferences.glyphID = 0 :FindGlyph() not return value

                                                                  If set   oIndesign.FindGlyphPreferences.glyphID = CID of oDocument.TextFrames(1).Characters(13) :FindGlyph()  return value

                                                                  Font name is Japanese

                                                                  Font name is English ,It is ok.

                                                                  • 31. Re: Finding Unassigned Glyphs
                                                                    mattAIM Level 1

                                                                    This is the best, juiciest addition to my InDesign workflow ever. Thanks so much Peter :-)
                                                                    I work with a lot of text books with Japanese and Chinese content. Our editors will often type script into a line of text (using InCopy) without the correct script font selected and I end up with 400pp+ docs riddled with these annoying, tiny pink boxes. Your script fixed this in minutes. Brilliant.

                                                                    • 32. Re: Finding Unassigned Glyphs
                                                                      PeteBaumgartner Level 1

                                                                      That's great

                                                                       

                                                                      Glad to help.

                                                                       

                                                                      I merged that script into our CopyFlow Gold extension at napsys.com

                                                                       

                                                                      All the best,

                                                                       

                                                                      Pjb

                                                                       

                                                                      Sent from my iPhone