15 Replies Latest reply on Jul 12, 2012 9:07 AM by ecto13

    Indesign 5.5, Book Production, XML Workflow & Digital Publishing

    luchty Level 1

            

      So here's the deal. I currently work for a publishing company. We create all of our print books in house. Here's an example of the production flow:

      >Obtain the manuscript from the author (in Word)

      >Do a sample page layout (in InDesign) (then approve)

      >Pour the (tagged) Word Doc into InDesign 5.5 and start; formatting, applying styles etc.

      >Proofed & Edited

      >Changes

      >Indexed

      >Back to the author

      >Review, Proof & Edited

      >Final • send files

      >Then take the InDesign files and convert them into epub/ebooks

       

      I know this is a loooooong, archaic, slow, drawn-out, old-fashion way to produce a book and then publish it in other mediums. Hence why I've been doing a boatload of research into XML publishing workflows. Here are some of the readings I've found:

       

      http://publishinggeekly.com/2010/11/indesign-xml-rope-of-sand/

      http://blog.gilbertconsulting.com/2010/05/preparing-indesign-file-for-xml.html

      http://www.xml.com/pub/a/2004/08/04/indesign.html

      http://projects.nordsueddesign.de/sidecarxml/

      http://tv.adobe.com/watch/instant-indesign/creating-a-basic-xmlready-template/

      http://adi-mps.com/Solutions/BookandJournal/TypesettingandXMLworkflows.aspx

      http://www.actualtech.com/products.php

      http://carijansen.com/2011/02/25/indesign-xml-part-2/

      http://answers.oreilly.com/topic/899-how-to-import-index-entries-from-word-to-indesign/

      http://www.adobe.com/products/indesignserver/features.html

      http://book.pressbooks.com/chapter/tools-of-the-digital-workflow-brian-oleary

      https://docs.google.com/viewer?a=v&q=cache:6PQl_YrI660J:wwwimages.adobe.com/www.adobe.com/ content/dam/Adobe/en/products/indesign/pdfs/indesign_and_xml_technical_reference.pdf+&hl=e n&gl=us&pid=bl&srcid=ADGEESioUKC_hoL8j7YQ4LUD3r_AC3OBOOaJZ35_ZoEKB1pNwHYFP4wO4XAcp-y_IVsr0 Vap8rS-Bg8LRDAq9B0EK6w12YG_9qMZ_ukguHJk8McNcJcxPc3p4aiyti__y67IzarsjBGa&sig=AHIEtbTakX7RTS JWjx-HWlfO7EUR81VMfA

       

      Now with all those links in mind. I've come to a conclusion it's not too—too simple in creating a XML workflow. However, since I am the sole party that does the ebook conversion and a lot of the book layouts, it's frustration to have such an ol' school system in place. I would love a centralized area where if a document was updated it would update throughout all branches of media etc.

      I feel we have the capabilities ie: MySQL server, MacServer, InDesign 5.5, InCopy, Word etc. I might ad, I'm also the creative department's IT & tech support. Without sounding cocky or arrogant, I consider myself an advance to expert Mac user and I have confidence I could/can setup such database. I've done a few tests just with generic Word files exporting them to XML files and importing the information into InDesign,,, with no 'real' success.

      So, does ANYBODY have a suggestion or a "How To Guide" or a step by step in how they setup their workflow?

      Any and ALL input would be greatly appreciated! Heck, I love learning advance concepts and new skills, so if anybody has a similar situation or scenario maybe we could/can work on this together.

      Thanks, T—

        • 1. Re: Indesign 5.5, Book Production, XML Workflow & Digital Publishing
          Kasyan Servetsky Level 5

          Here are some interesting resources on using XML in InDesign:

          Designer's Guide to Adobe InDesign and XML by James Maivald and his course on Lynda.com

          XML Publishing with Adobe InDesign by Dorothy Hoskins

          1 person found this helpful
          • 2. Re: Indesign 5.5, Book Production, XML Workflow & Digital Publishing
            Fred Goldman Level 3

            We are in a very similar situation as yours. A small publishing house looking to publish to multiple mediums and we do everything in-house. Our workflow is almost identical to yours, only I write scripts to automate a lot of the style application when the Word file is brought into InDesign.

             

            We grappled with the same issues as yours and finally came to the conclusion that the only way we could keep all the information we needed (formatting etc.) and get the content to multiple mediums was to apply the xml in InDesign (automatically with scripting) and then export that to xml and manipulate that xml.

             

            Whether you need to go about the same route depends on how complicated you publications are. I would have to see them to know for sure, but there are some publications where an xml->multiple mediums works and there are others that need xml->InDesign->multiple mediums.

             

            The ones that have tried xml->multiple mediums are companies like TypeFi, WoodWing, and Quark. If your publications fit into that model you can go that route and look to print as just another way of publishing. If not, you most likely have to some kind of tagging in InDesign and then try and repurpose that xml for whatever is necessary.

            • 3. Re: Indesign 5.5, Book Production, XML Workflow & Digital Publishing
              luchty Level 1

              Kasyan, Thanks for the info. I've downloaded both of those publications and started to read them. I guess my hang up,, or I should say my 'apprehension' (and this is kinda geared for Fred) — our books are pretty complex or artistic. NOT just bare bones; <H1> <H2> <body> etc. They also have different and weird fonts. I'm gonna start doing some more and more research. Anybody else have ideas?

              • 5. Re: Indesign 5.5, Book Production, XML Workflow & Digital Publishing
                Joel Cherney Adobe Community Professional & MVP

                our books are pretty complex or artistic. NOT just bare bones; <H1> <H2> <body> etc. They also have different and weird fonts. I'm gonna start doing some more and more research. Anybody else have ideas?

                 

                I have done some vaguely related work - I've done a few  database -> XML -> InDesign projects, with varying degrees of heavy styling in the final product. Without knowing exactly what you're doing, I'll just say that a workflow relying upon importing XML into InDesign might be more trouble than it's worth, depending on the repetitiveness/cleanliness of both the source data and of the final product.

                 

                For what it's worth, the hand-cranked Word-based workflow you describe in your original post is what I do every day - when I'm not using a translation memory tool to achieve the same effect with a lot less work. However, even with non-complex non-artistic work, I put a lot of manual effort into any target language in which the format changes appreciably. For example, I'm doing a twenty-language job right now in which much of the format work of the Latin-script LTR languages happens automagically, but the SE Asian scripts and right-to-left languages (Arabic, Persian) are good ol' fashioned layouts by hand, like Mom used to make. They could be automated, but the effort put into automation does not pay off. (And I'm also doing another similar project where the complex artistic layout pretty much forces every single language to be laid out by hand, even when a database is doing the heavy lifting on the editorial-review-on-text part of the workflow.)

                1 person found this helpful
                • 6. Re: Indesign 5.5, Book Production, XML Workflow & Digital Publishing
                  Fred Goldman Level 3

                  I didn't necessarily mean something as basic as h1-h6 and other html tags. Some xml workflows are very robust that work with styles that allow you any combination of fonts, formatting etc. Typefi and Woodwing allow for a lot of flexibility in their solutions, but that doesn't mean there systems work for everyone.

                   

                  Where things become difficult is where you have mutiple stories that have to work together on one page or where text has to be "massaged" to make it fit on the page. These type of things become difficult to automate. And as Joel said, it doesn't always make sense cost wise to automate these things. However, now that ebooks are becoming popular the decision as to whether to automate or not is becoming a more costly decision.

                   

                  If going straight from xml->indesign->print is not possible without manual intervention then you should looking into getting the format needed out of InDesign either by tagging the InDesign document with xml or by exporting to epub, xhtml, and or idml and manipulating that with xslt and/or E4X.

                  • 7. Re: Indesign 5.5, Book Production, XML Workflow & Digital Publishing
                    John Hawkinson Level 5

                    Sorry for the delay in responding...I just couldn't get up the energy to deal with XML over the weekend.

                     

                     

                    Without sounding cocky or arrogant, I consider myself an advance to expert Mac user and I have confidence I could/can setup such database. I've done a few tests just with generic Word files exporting them to XML files and importing the information into InDesign,,, with no 'real' success.

                     

                    I think this absolutely a dead-end horrible workflow and you should not even be considering it.

                     

                    XML is complicated. XML is hard.

                    One program's XML output is not another program's XML input.

                    Not not not not not not not not not.

                     

                    Maybe, in some cases, an XML output from some program can be translated into an XML input to another program (as via XSLT), but it will typically be painful.

                     

                    InDesign's XML input is not good. It is very restrictive and essentially tied to a single story (which can have anchored frames, etc.).

                    Of course, you can write your own XML import using InDesign's extremely flexible scripting support and XML support for JavaScript (E4X).

                    Or you could produce IDML and use InDesign's IDML import instead of XML import.

                     

                    IDML is an InDesign XML format that completely describes the document. That means it is incredibly complicated to produce, and trying to produce a Word to IDML converter is basically like trying to rewrite part of InDesign. It will take man-years and it still won't be very good.

                     

                    You haven't really told us why you want XML and what you think it wrong with your original workflow. You've described it as old-school, but haven't talked about what the limitations are that bother you, or what you hope to gain.

                     

                    Your workflow seems extremely linear, with the possible exception of the Author review/proof stage.

                     

                    Your narrative seemed to focus on getting Word -> InDesign to be Word -> XML -> InDesign.

                    I don't know why you would want to do that.

                    Do you have other consumers of Word's XML output?

                     

                    If you just want XML because you think it's modern, think again. XML sucks.

                     

                    What are your goals?

                    • 8. Re: Indesign 5.5, Book Production, XML Workflow & Digital Publishing
                      luchty Level 1

                      Joel, Thanks for you input. You also bring up another good point! We do a lot of foreign font based books ie: TransIndic, Sanskrit etc. along with other special characters. I can/could foresee this being a big problem.

                      John, basically my/our goal(s) are quicker turnarounds for books. And to make a easier way to redistribute the "guts"/books information ie; epubs, digital publishing and recycle for our web team. I would love the production editor or production artist to be able to make corrections in the XML file and have it be instant, real-time updating. The ol' school method I'm referring to,, we physically PRINT out the whole book in each step of our production rounds (keep in mind a book usually goes through 3-6 rounds before final, print). It's painful and the slow process is crippling our book workflow. So, I guess that is my goal. To make our book workflow less like a Yugo and more like a Bugatti (better than Ferrari).

                      • 9. Re: Indesign 5.5, Book Production, XML Workflow & Digital Publishing
                        John Hawkinson Level 5

                        T:

                        I'm sorry to repeat, but can you please be more specific:

                        John, basically my/our goal(s) are quicker turnarounds for books. And to make a easier way to redistribute the "guts"/books information ie; epubs, digital publishing and recycle for our web team. I would love the production editor or production artist to be able to make corrections in the XML file and have it be instant, real-time updating.

                        Which XML file? The Word XML export?

                        How do you do EPUBs and Digital Publishing now? Do you not leverage InDesign for these? If you don't, tell us more -- which file goes where? If you do, how does XML help you here?

                         

                        The ol' school method I'm referring to,, we physically PRINT out the whole book in each step of our production rounds (keep in mind a book usually goes through 3-6 rounds before final, print). It's painful and the slow process is crippling our book workflow. So, I guess that is my goal. To make our book workflow less like a Yugo and more like a Bugatti (better than Ferrari).

                        I guess I'm confused what this has to do with your potential XML workflow. Presumably your printing has reviewing benefits...what would you get rid of, why?

                        • 10. Re: Indesign 5.5, Book Production, XML Workflow & Digital Publishing
                          Joel Cherney Adobe Community Professional & MVP

                          My XML-driven-workflow experiences have been comparatively easy because I've never needed to write a lick of XSL. I just say to the database maintainers, "Your XML output is imperfect here, here, and here. Fix immediately or the whole project goes kaput." And then, the DB boys fix it.

                           

                          John's experiences are obviously less smooth than mine; however, I don't think that John can do what I do and just pass the buck to the database maintainers. He actually has had to write XSL transforms to chivvy dirty XML into place-ability. (I've done it once or twice, but never under any kind of delivery deadline pressure.) If I didn't have expensive tools to do the XML-tag-handling heavy lifting for me, and the DB-to-XML stuff wasn't handled by a whole team of people who knew database development as well as I knew translation memory management tech, I would not attempt this kind of workflow.

                           

                          Furthermore, John points out that XML input into InDesign is very restrictive. It's the bees' knees for things like dictionaries, that involve complex layouts that are very linear. We have no clear idea what you're working on, but it doesn't sound like you're making complex, artistic glossaries. If I'm wrong, then XML might be the way to go for you.

                          Joel, Thanks for you input. You also bring up another good point! We do a lot of foreign font based books ie: TransIndic, Sanskrit etc. along with other special characters. I can/could foresee this being a big problem.

                           

                          Uh... TransIndic? From Linguists' Software, the foundry I love to hate? It'll be a massive problem. Last time I checked, they'd actually threaten you with lawyers if you even embedded their fonts into PDFs. If you were serious about developing some kind of managed multilingual editorial workflow, this would be a show-stopper.

                          • 11. Re: Indesign 5.5, Book Production, XML Workflow & Digital Publishing
                            Joel Cherney Adobe Community Professional & MVP
                            Uh... TransIndic? From Linguists' Software, the foundry I love to hate? It'll be a massive problem. Last time I checked, they'd actually threaten you with lawyers if you even embedded their fonts into PDFs. If you were serious about developing some kind of managed multilingual editorial workflow, this would be a show-stopper.

                             

                            Whoops, didn't mean to post - who thought it would be a good idea to map Ctrl-S to "make a post on the Internet?"

                             

                            Anyhow, if you want some sort of database into which people contribute edits to texts set in non-Unicode fonts from Linguists' Software, you're looking at some expensive font licensing and a lot of work. But if you're using their fonts, then you are quite likely doing some kind of work that requires glyphs not yet in Unicode - perhaps some kind of scholarly or religious publication? Or maybe LS has gotten with the 21st century (and converted their broken-encoding Type 1 fonts to Unicode OTF) and I'm unaware of it.  If that's the case, ignore my wild head-shaking and hand-waving.

                            • 12. Re: Indesign 5.5, Book Production, XML Workflow & Digital Publishing
                              John Hawkinson Level 5

                              John's experiences are obviously less smooth than mine; however, I don't think that John can do what I do and just pass the buck to the database maintainers. He actually has had to write XSL transforms to chivvy dirty XML into place-ability. (I've done it once or twice, but never under any kind of delivery deadline pressure.) If I didn't have expensive tools to do the XML-tag-handling heavy lifting for me, and the DB-to-XML stuff wasn't handled by a whole team of people who knew database development as well as I knew translation memory management tech, I would not attempt this kind of workflow.

                              To be fair, the reality is somewhat different.

                               

                              We don't have a lot of automation for InDesign's inputs, and what we do have generally uses InDesign Tagged Text rather than XML.

                              We do have significant autmation based on InDesign's IDML output.

                              But I have messed around with InDesign's XML import fairly significantly, both to learn how it works and also to assist people on this forum who come with inquiries that nobody seems to want to touch with a 10-foot pole...

                              • 13. Re: Indesign 5.5, Book Production, XML Workflow & Digital Publishing
                                luchty Level 1

                                   

                                T:

                                I'm sorry to repeat, but can you please be more specific:

                                John, basically my/our goal(s) are quicker turnarounds for books. And to make a easier way to redistribute the "guts"/books information ie; epubs, digital publishing and recycle for our web team. I would love the production editor or production artist to be able to make corrections in the XML file and have it be instant, real-time updating.

                                Which XML file? The Word XML export?

                                How do you do EPUBs and Digital Publishing now? Do you not leverage InDesign for these? If you don't, tell us more -- which file goes where? If you do, how does XML help you here?

                                 

                                The ol' school method I'm referring to,, we physically PRINT out the whole book in each step of our production rounds (keep in mind a book usually goes through 3-6 rounds before final, print). It's painful and the slow process is crippling our book workflow. So, I guess that is my goal. To make our book workflow less like a Yugo and more like a Bugatti (better than Ferrari).

                                I guess I'm confused what this has to do with your potential XML workflow. Presumably your printing has reviewing benefits...what would you get rid of, why?

                                 

                                 

                                 

                                 

                                Well, I'm not sure what XML file. But, how I/we currently do EPUB's and digital publishing.... is I take the 325pg InDesign (highly stylized) 'book' and I strip it down to bare-bones styles (nothing too complex). Then I export an epub from InDesign and then I tweak and massage the file using Sigil. (Did I mention; all of these steps suck). Then I make a different version for every device out there ie: ipad, kindles, androids etc. As far as the digital side goes,, I take and package the book into a final PDF (edit the metadata 'properties') then delivery it to our dedicated web department according to their specs.

                                 

                                So maybe I'm naive or unedumacated, but from what I've read about XML I understand it to be a centralized "HUB" of a wheel, then it feeds outward to different spokes of the wheel. And if something is changed local, it will change globally (similar to css etc.) but on the content side.

                                Am I totally lost?

                                Or am I mislead in what I'm trying to achieve?

                                Thanks, T—

                                • 14. Re: Indesign 5.5, Book Production, XML Workflow & Digital Publishing
                                  John Hawkinson Level 5

                                  Well, I'm not sure what XML file. But, how I/we currently do EPUB's and digital publishing.... is I take the 325pg InDesign (highly stylized) 'book' and I strip it down to bare-bones styles (nothing too complex). Then I export an epub from InDesign and then I tweak and massage the file using Sigil. (Did I mention; all of these steps suck). Then I make a different version for every device out there ie: ipad, kindles, androids etc. As far as the digital side goes,, I take and package the book into a final PDF (edit the metadata 'properties') then delivery it to our dedicated web department according to their specs.

                                  I'm not sure what you mean by "stripping down."

                                  And certainly no idea what particular kind of tweaking and massaging you do in Sigil.

                                  Perhaps you could be more step-by-step about this.

                                  I'm also not sure what the "digital side" means -- how is PDF any more or less digital than EPUB?

                                   

                                  Are you aware that a future version of InDesign is expected to have a "liquid layout" feature designed to deal with different versions for different devices. See http://indesignsecrets.com/dynamic-liquid-layout-in-indesign-max-sneak-peek.php.

                                   

                                  But in any case, that's not here today and who knows what form it will be in tomorrow.

                                   

                                  Anyhow, I would say that XML is probably not helpful to you here. You can automate your stripping in InDesign scripting, and you can automate your teaking and massaging of the EPUB in the XML/HTML manipulation language of your choice (umm), and I don't know what you need to do with the PDF.

                                  So maybe I'm naive or unedumacated, but from what I've read about XML I understand it to be a centralized "HUB" of a wheel, then it feeds outward to different spokes of the wheel. And if something is changed local, it will change globally (similar to css etc.) but on the content side.

                                  Am I totally lost?

                                  Or am I mislead in what I'm trying to achieve?

                                  I'm afraid you are both lost and mislead. Sorry.

                                  XML is an extremely loose specification of a file format that can have millions of different tags and ways to represent data within that format.

                                  It certainly does not have an inherent automatic updating. While you can certainly set up an automated process to transform one XML format into another, that is not typically automatic.

                                   

                                  It's not necessarily true that XML is a centralized hub, either. If InDesign set to import an XML file which fully describes your document, or to export an XML file which fully describes your document, then you could potentially drive other things from either of those XMl files. But getting to a point where either those situations is true is nontrivial.

                                  But EPUB files are a type of XML (but they have other things with them, too, like CSS).

                                  You could conceivably use any one of those 3 (actually 4) XML files to drive some other XML workflow.

                                   

                                  But you haven't really suggested anything that you would use XML for, other than perhaps importing it into InDesign. But what would produce that XML? I don't think you would find that the Word XML was suitable for importing into InDesign. And even if it were, why would going from Word -> InDesign via XML be any better than with .doc or .docx or .rtf, as you [presumably] do now?

                                  • 15. Re: Indesign 5.5, Book Production, XML Workflow & Digital Publishing
                                    ecto13

                                    For our eBook creation we simply export to eBook from InDesign, tweak it with Sigial, use Calibre to add metadata and an interactive TOC (generated by the chapter styles), re-export from Calibre in any format we want.