22 Replies Latest reply on Dec 12, 2018 1:44 AM by TᴀW

    2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?

    TᴀW Adobe Community Professional & MVP

      Would anybody know why the following snippet throws an error when the ESTK target is 2019, but works fine when the ESTK target is 2018 (or ESTK itself)?

      Is ASCI 22 in fact not kosher for an XML text node?

       

      a = "apples";
      b = "pears";
      c = String.fromCharCode(22); // ASCII 22 is the character InDesign uses as a table placeholder in a textframe.
      x = <food>
      <fruit>{a}</fruit>
      <fruit>{b}</fruit>
      <fruit>{c}</fruit>
      </food>;
      alert(x);
      

       

      When targetting 2019 (even though the 2019 DOM is of course not touched by this script), ESTK throws an "XML error in line 4", and highlights line 7.

      But it's fine if the target is 2018.

      Am I missing something obvious?

       

      Thanks,

      Ariel

        • 1. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
          [Jongware] Most Valuable Participant

          Is your code to create XML on the fly inside InDesign? How does that table (without any data inside!) then appear? Perhaps it's a bug that you could actually specify it in earlier versions and that has been fixed ...

          1 person found this helpful
          • 2. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
            TᴀW Adobe Community Professional & MVP

            It's the other way round. This is part of a script that creates an XML object from the contents of text frames.

            Until now it worked fine. But in 2019 it started throwing an error in case the text frame contained a table.

            So, for example:

            t = myTextFrame.contents;

            x = <myxml><item1>{t}</item1></myxml>

             

            If myTextFrame contains a table, t will contain ascii 22, and the script will crash on line 2. In 2019. But not in earlier versions.

            • 3. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
              TᴀW Adobe Community Professional & MVP

              To put it in a nutshell, the following 1-line script throws an error if the target in ESTK is InDesign 2019, but not if it's ESTK, or InDesign 2018.

               

              x = <myxml>{String.fromCharCode(22)}</myxml>

               

              To me it seems very strange the the Javascript interpreter should be affected by the target-app dropdown in ESTK, especially considering that this 1-liner doesn't actually touch the target application in any way.

              • 4. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                Trevorׅ Adobe Community Professional

                Interesting one

                var x, charCode;
                for (charCode = 0; charCode < 10000; charCode++) {
                    try {
                        x = <myxml>{String.fromCharCode(charCode)}</myxml>
                    } catch (e) {
                        $.writeln(charCode + '\t' + String.fromCharCode(charCode) + '\t' + e);
                    }
                }
                

                On Illustrator 2019 no problems.

                On InDesign 2019

                0

                1   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                2   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                3   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                4   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                5   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                6   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                7   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                8   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                11   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                12   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                14   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                15   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                16   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                17   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                18   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                19   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                20   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                21   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                22   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                23   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                24   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                25   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                26   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                27   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                28   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                29   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                30   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                31   SyntaxError: XML error in line 1
                - Not well-formed (invalid token)

                 

                Solution / Workaround ?

                xmlRegex = /[\x01\x02\x03\x04\x05\x06\x07\x08\x0b\x0c\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f]/g;
                myXMLSafeContents = myContents.replace(xmlRegex, function(r){return '#!@' + r.charCodeAt() + '#!@@';});
                
                2 people found this helpful
                • 5. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                  Laubender Adobe Community Professional & MVP

                  Hi Trevor,

                  very strange, indeed.

                   

                  If the target is the ESTK: No error message.

                  With InDesign CC 2019: All the errors that you are listing.

                  No error messages with e.g. InDesign CC 2018.1, PhotoShop CC 2019.

                   

                  Regards,
                  Uwe

                  1 person found this helpful
                  • 6. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                    Trevorׅ Adobe Community Professional

                    Regarding the validity of these characterless see Valid characters in XML - Wikipedia

                    It could explain why they work in one version and not the other but still it's strange.

                    1 person found this helpful
                    • 7. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                      Trevorׅ Adobe Community Professional

                      N.b. The regex was a bit careless should not include \x09\x0a\0d ! it try edit it.

                      • 8. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                        Martinho da Gloria Level 1

                        Different approach, but this should work.

                        var a = ["apples",  "pears", String.fromCharCode(22)];

                        var x = <food/>;

                        for (var idx = 0; idx < a.length; idx++) {

                            var f = <fruit/>;

                            f.appendChild(a[idx]);

                            x.appendChild(f);

                        }

                        $.writeln(x.fruit[2].toString().charCodeAt());

                        x.toXMLString();

                        2 people found this helpful
                        • 9. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                          TᴀW Adobe Community Professional & MVP

                          Hi Trevor,

                          Thanks. Interesting that you were even able to wrap this in try-catch. Were you running the script in InDesign? When I tried to run it in ESTK with ID2019 as target, even try-catch didn't help. (Actually, maybe I didn't check my debug options, but I think I did tell it not to break.)

                          • 10. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                            TᴀW Adobe Community Professional & MVP

                            So which app is bugging out? It's got to be ID2019, no? Or are we saying that's the only one that's got it right?

                            • 11. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                              TᴀW Adobe Community Professional & MVP

                              I saw that, but it's not clear to me which variety of XML our Ecmascript is based on.

                              • 12. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                                TᴀW Adobe Community Professional & MVP

                                Good idea. That does work around the bug.

                                Still wonder what's going on with the ESTK parser in ID2019, though.

                                • 13. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                                  Laubender Adobe Community Professional & MVP

                                  TᴀW  wrote

                                   

                                  Hi Trevor,

                                  Thanks. Interesting that you were even able to wrap this in try-catch. Were you running the script in InDesign? When I tried to run it in ESTK with ID2019 as target, even try-catch didn't help. (Actually, maybe I didn't check my debug options, but I think I did tell it not to break.)

                                  Hi Ariel,

                                  I did my tests with the ESTK CS6 3.8.0.12 and also with ESTK CC 4.0.0.1 with target InDesign CC 2019 14.0.1.209 on Windows 10 Pro. Trevor's code with try-catch worked like it should. Had exactly the same results as Trevor is showing in reply 4. Maybe it will turn out differently, try-catch not working, on OS X ?

                                   

                                  TᴀW  wrote

                                  So which app is bugging out? It's got to be ID2019, no? Or are we saying that's the only one that's got it right?

                                   

                                  Hard to tell. To my tests only InDesign CC 2019 is showing the new behavior.

                                  In the meanwhile I also tested InCopy CC 2019 and Bridge CC 2019 as target. No issue even with InCopy.

                                   

                                  Regards,
                                  Uwe

                                  2 people found this helpful
                                  • 14. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                                    TᴀW Adobe Community Professional & MVP

                                    Checked again, you're right indeed. Try-catch does work. I must have had the "do not break on guarded exception" unchecked.

                                    So, this is mysterious. Martinho's answer below is a decent workaround, but still.

                                    Maybe worth posting this on the beta testing as a bug, and see what the Adobe engineers have to say...

                                    • 15. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                                      Trevorׅ Adobe Community Professional

                                      Hi Uwe

                                       

                                      Very interesting about InCopy which my understanding was that it's basically InDesign core with a different set of plugins and UI.

                                       

                                      Hi Ariel

                                       

                                      Martinho's method I think is the way to go, XML literals E4X had not been supported in mainstream JS environments for years and I can't see it holding out for long in AdobeLand. Using methods like appendChild is going to make porting to the new environments a lot more simple as those methods are much more similar to what is going to be available.

                                      You can see what the engineers come up with but it just a matter of time before it breaks again forever even if they do fix it.

                                      • 16. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                                        TᴀW Adobe Community Professional & MVP

                                        You're probably right. OTOH. frameworks like React are bringing a very similar syntax back in again...

                                        1 person found this helpful
                                        • 17. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                                          Trevorׅ Adobe Community Professional

                                          Yes, that's a very good point about react.

                                          • 18. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                                            Harbs. Level 6

                                            FWIW, I filed a bug report on some similar XML some weeks ago.

                                             

                                            Unfortunately, I have no access to that particular bug anymore, so I can't update it with a reference to this thread. The bug number is 4211924.

                                             

                                            Harbs

                                            • 19. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                                              Marc Autret Level 5

                                              Hi all,

                                               

                                              TaW made a crucial point:

                                               

                                               

                                              (…)

                                              To me it seems very strange the the Javascript interpreter should be affected by the target-app dropdown in ESTK, especially considering that this 1-liner doesn't actually touch the target application in any way.

                                              Serious question indeed. Provided that the target app—i.e. its particular DOM—is not involved in the reported error, why should ID CC 2019 behave differently, versus ID CC 2018 or even PS CC 2019?

                                               

                                              This enigma led me to investigate and here is what I found:

                                               

                                              1. The XML parser which ExtendScript relies on is (very likely) the Expat library designed by James Clark:

                                              libexpat/expat at master · libexpat/libexpat · GitHub

                                              What allows me to make this assumption is that typical XML tokens and error strings appear in ExtendScript's core library (ScCore.dll) and most of them are clearly connected to Expat parameters.

                                               

                                              2. Expat is “an XML 1.0 parser”, as stated in its documentation and confirmed in its source code. For example, the header asciitab.h has the folowing lines

                                               

                                              Non-XML.PNG

                                              which indicates that almost all control codes from 0x00 to 0x1F are considered NONXML—and then should be reported as invalid by the tokenizer.

                                               

                                              As already observed by my colleagues, the only control codes allowed in XML 1.0 are U+0009, U+000A, U+000D. Hence, the code 0x16 (decimal 22) which TaW has inserted in its XML stream should have failed from the very beginning, in any target app.

                                               

                                              3. However, Expat has, or had, various bugs that made it more or less permissive regarding the XML spec. Many updates and fixes are reported in the changelog from release 1.95.0 (Sept. 2000) to 2.2.6 (Aug. 2018.) I do not understand those technical details, but sentences like “Fixed UTF-8 decoding bug that caused legal UTF-8 to be rejected”, “Check that a UTF-16 encoding in an XML declaration has the right endianness”, or “Mass-fix compilation for XML_UNICODE_WCHAR_T”, etc. sound very much in connection with our topic.

                                               

                                              Also, depending on how Expat is configured when compiling the lib, I suppose that the client can get the tool in various flavors. Cf. the section “Configuring Expat Using the Pre-Processor” in the documentation.

                                               

                                              4. Now, let's go back to the original question: how is it possible that ID CC 2019 differs from ID CC 2018 regarding XML parsing at the ExtendScript level? My guess is that they may not use the same Expat version (in ScCore.dll.) The following screenshots strongly support that view:

                                               

                                              CC-2018-Expat.png

                                               

                                              CC-2019-Expat.png

                                               

                                              As you can see, InDesign 13.x embeds ExtendScript with Expat 1.95.7 while InDesign 14.x uses Expat 2.2.5.

                                              This answers at least a part of the question: we know that ExtendScript is not the same in both apps, at the very core level.

                                               

                                              As the bonus, I also gave a look at ScCore.dll in Photoshop CC 2019.

                                              Guess what, it still uses Expat 1.95.7, as does ID CC 2018.

                                               

                                              PS-2019-Expat.png

                                               

                                              Maybe just a coincidence, but it fits very well the symptoms!

                                               

                                              Best,

                                              Marc

                                              2 people found this helpful
                                              • 20. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                                                Laubender Adobe Community Professional & MVP

                                                Hi Marc,

                                                also test with target InCopy CC 2019 where I cannot see the same behavior InDesign CC 2019 is showing.

                                                If I am looking for expat_ in ScCore.dll of InCopy CC 2019 I can see version expat_2.2.5.

                                                 

                                                Maybe there is more to it than the version of expat alone.

                                                 

                                                expat_2.2.5-with-InCopy-ScCore.dll.PNG

                                                 

                                                Regards,
                                                Uwe

                                                1 person found this helpful
                                                • 21. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                                                  Marc Autret Level 5

                                                  Damn! Expat was my ideal suspect…

                                                  So there is something else, or something more.

                                                   

                                                  [Noob question: I don't know how InCopy works, or whether it may interact with ID at some level… Are the apps fully disconnected?]

                                                   

                                                  @+

                                                  Marc

                                                  • 22. Re: 2019 bug? This ECMAScript XML snippet works in 2018 but not 2019?
                                                    TᴀW Adobe Community Professional & MVP

                                                    Until they fix the bug base, nobody can see anything. I also filed a bug, but following Marc's analysis perhaps the bug was in the previous versions.