Correct procedure to re-import XML?

[JustyR]

Hi all,

Just wondering what the correct procedure is to RE-import an XML file into an InDesign document.

I have an INDD template and XML file that both work well together.

They help me create a booklet of 300+ biographies (pic, name, company, biog...etc.)

When I first import the XML, it all flows in correctly, styles itself and imports all images. All is well.

If that XML file is changed (due to client updates), and I re-import; I'll find that what is deleted in the XML, can still show up in the InDesign document. It's as if InDesign is remembering what was there and keeping it. Or worse, John Smith's Twitter handle shows up in Jane Smith's Twitter handle (even though she doesn't have one).

Does a re-import work this way if you don't delete the spreads and start afresh with a clean spread, clean ROOT and fresh import?

[JustyR]

After lots of trial and error, I've found out the following:

To make it easier to visualise, imagine you have an Excel spreadsheet and a Word doc. Excel is your XML, Word is your INDD.

...

Jane Smith is in Row 2 and she has an e-mail address in cell E2

John Smith is in Row 3 and he has NO e-mail address in E3 (a blank cell)

...

When you import that Excel into Word for the first time, everything shows how it should.

You now delete Row 2 (Jane Smith, she's got to go!) in the EXCEL.

John Smith automatically moves up to Row 2. Remember, he doesn't have an e-mail address (E2 is now blank).

In Word, you re-import this new Excel. Jane Smith is gone, but her e-mail address has stayed in Word and is in with John Smith's details.

What this means for an XML and INDD workflow is that there's no quick way to re-import an updated XML (PLEASE tell me if there is).

The quickest way I've found is to delete all but the first element in the Structure pane. Alt-click the right arrow next to your first element (not Root). And double-click each text snippet and hit backspace. If you have any hrefs for images, then single-click and hit backspace. You're taking everything back to when you first imported all your data, which could be days ago.

This gives you blank frames and tags on your page (any images will stay, but don't worry about it). Now import the updated XML. It means that you don't need to delete spreads and copy/paste original frames/tags from a template.

It's probably about a minute's worth of faff to import an updated XML without mucking everything up.

If it's easier than this, please let me know. I hazard a guess this is as good as it gets.

[MW Design]

I don't know the structure of your XML nor the contents of the elements, but you should be able to select the root and reimport. That said, I generally link to the XML and if/when changes are made I update the link. I get no "ghost" elements left over after deletions and additions are properly included.

Mike

[JustyR]

That's useful to know.

I'm using anchored and linked text frames and I'm wondering if this is the problem.

[MW Design]

I don't know if the anchored items are an issue or not as I don't remember having used them in an XML work-flow in the past. As regards linked text frames, that's pretty standard.

Mike

[JustyR]

Do you happen to have a copy of A Designer's Guide to Adobe InDesign and XML? I've found the exact same 'ghosting' problem in a sample indd and xml supplied with the book.

I import an XML file into their template. All comes in lovely. I go back to the original XML file and delete some text between tags. Re-import in InDesign and it's as if nothing has changed. The text is still there.

If you have the time, would be cool if you could test this yourself.

[JustyR]

Some further testing gives another solution...

Let's say this is my XML file:

<root>

<person>

<name>John</name>

<company>BigShop</company>

<email>xxATxxxx.xxx</email>

</person>

</root>

I've imported the following into InDesign and each 'person' is in their own text frame.

John

BigShop

xxATxxxx.xxx

If I now go back to that XML file, and delete the e-mail address (but not the elements):

<root>

<person>

<name>John</name>

<company>BigShop</company>

<email>xxATxxxx.xxx</email>

</person>

</root>

InDesign still keeps that e-mail address.

John

BigShop

xxATxxxx.xxx

HOWEVER, if I delete the elements as well:

<root>

<person>

<name>John</name>

<company>BigShop</company>

</person>

</root>

...and turn on DELETE ELEMENTS...on import:

...it works. The e-mail address is gone.

The major downside is that should that element go back in, it won't come in and is essentially again a form of this ghosting problem.

[MW Design]

I would need to see a screen shot of the page and showing the structure panel to even hazard a guess as to what/why you are getting that result. For instance, I also don't know why they are in separate frames. Is this a necessity?

Mike

[JustyR]

They're in frames because of the design.

Here's a screenshot of the structure and page. This is actually a template that came with that XML book I mentioned.

If I delete the word "Jim" from the XML doc in Efirstname, and re-import, you can see that "Jim" is still there. Even in the Structure pane.

[MW Design]

I think I have the book in a box. I'll take a look tomorrow (well, today...but after a nod-off).

What chapter?

[JustyR]

Chapter 4?

P111

Sample Project: 6-Up Business Cards

Using "6cards.indt" from the sample files

[Laubender]

Hi,

fortunately I have the book at hand and all the chapter materials available 🙂

The thing is, if you do not provide contents the imported XML will not remove the contents that is already there.

Before:

<Efirstname>Jim</Efirstname>

And "First" is in the InDesign template document surrounded by the tags.

"First" will be replaced with "Jim" after import.

Now close the document without saving.

If you now edit the XML:

<Efirstname></Efirstname>

And do an XML import on the reopend template, "First" will not be removed.
The XML has nothing to add and leaves the document between the two tags unchanged.

So you could change the template and remove the text between the tags to plan ahead for situations where the contents is empty.
E.g. no name or address provided.

After importing XML:

Regards,
Uwe

[Laubender]

But your question revolves around a re-import of XML data.

So to start from scratch every time after the XML data has to be changed might be unsatisfactory:
Maybe you edited your layout after creating the first import and there is a last-minute change in the source of the XML.

Then one solution could be not to provide nothing as in my example above.

How about providing contents, but in a special way?

Maybe a special string, something like:

<Efirstname>$E$</Efirstname>

And your used paragraph style is reacting with a GREP Style like: \$E\$ and an applied character style named "Invisible", that will format this string to 0.01 point text size with fill and stroke color to "None"?

A dirty trick, I know, that could backfire if one does not remove the strings before exporting to e.g. PDF with read aloud abilities.

But wouldn't that be a solution for print-only material?

Regards,
Uwe

[JustyR]

Thanks for helping out.

My trial and error over a few days tallies with what you've suggested.

Quite why InDesign has this limitation, when you can create a link to an XML file as well. It's been really difficult to find information about this 'problem'. I'd have thought it would have come up time and time again.

My first workaround option, as you say, is to strip the InDesign doc back to basics for another import. This is fine for my own circumstances as all the content is in linked frames and is the only part of the document that uses the XML process. But further down the line as I use XML more and more for other projects, this might be a problem, as you say.

Your 'dirty trick' could work. Forcing InDesign to see the element as filled, overwriting content. A grep style in the paragraph style hides it (sort of). I'd have to incorporate that Grep style into all paragraph styles though.

Automating the insertion of $E$ into the XML could be achieved after the initial conversion of Excel data to XML tagged data; via a grep find/replace.

Something like...(this is in BBEdit or TextWrangler)

Find:

(.>)(</.)

Replace:

\1\$E\$\2

This:

<Employeeid></Employeeid>

Becomes this:

<Employeeid>$E$</Employeeid>

Thanks again for your help as it's good to know it's not me, it's InDesign that's the problem. I might flag it as a bug or feature request with Adobe.

[Laubender]

Hm,
I'm not sure it's a bug. More a feature limitation 😉

Wishform - Adobe InDesign

Btw. That "special string" "$E$" is no recommendation.
Maybe you could exchange it with a better one.

Regards,
Uwe

[Laubender]

Speaking about hiding contents:
How about using "Conditional Text"?

But that would require a script maybe to apply a condition to all text that matches a certain substring.

Regards,
Uwe

[JustyR]

Could be useful but does take me down the realms of making the process more and more complex.

[Laubender]

Yes. It would be much easier, if character styles would be able to store an applied condition or an applied set of conditions. Also something for the Wishform…

Regards,
Uwe

[JustyR]

Ok, more testing done. Some of these comments are for me for future reference, but also to help others.

If you do the following, see what happens when it's imported a second time:

1- You make your changes in the InDesign doc	XML doc imported again - what you see in InDesign
1) You remove "Jim" from between the Efirstname tags	Jim comes back in, undoing your INDD change
2) You change "Jim" to "James" within the Efirstname tags	James reverts back to Jim, undoing your INDD change

2- You make your changes in the XML doc	XML doc imported again - what you see in InDesign
1) You remove "Jim" from between the Efirstname tags	Jim is still there, undoing your XML change
2) You change "Jim" to "James" within the Efirstname tags	Jim now becomes James

I still fail to see how this behaviour can be helpful. But it's early days with InDesign and XML for me.

Moving on...

We now know that making amendments to text in InDesign is of no use if not replicated in the XML doc. We also know that InDesign will only update text if it sees some already on your page between tags.

With that in mind, Uwe's suggestion of Conditional Text can help.

In the XML doc, a Grep query can convert empty elements to filled ones:

Find query	Finds this	Change query	Changes to this
(.>)(</.)	x></x	\1\$E\$\2	x>$E$</x

So this:

<name></name>

Will become this:

<name>$E$</name>

Now when the XML is imported into InDesign again, anything previously blank, is now filled with faff ($E$). Highly unlikely to occur anywhere else.

Create a Conditional Text condition in InDesign called "Empty"

Turn its visibility off

Upon each import of the XML, run a Grep Find/Replace (see the screenshot) that looks for $E$ and applies the Empty condition.

When complete, all those instances should disappear from view.

If the XML is updated yet again, they will reappear, so simply run the Grep Find/Replace that applies the Empty condition.