12 Replies Latest reply on Jan 17, 2015 4:23 PM by wildcoastpub

    Merge data creating duplicate listings

    wildcoastpub

      I have a csv database with about 580 records. I've created a template with the correct fields in inDesign, loaded the csv file using merge data, then created the merged document. I get 580 listings of one item like the image below. If there is an errant symbol in the csv database causing this, it doesn't reflect in an error in a preview. In fact, all 580 records can be previewed, copied and pasted into another document and displayed error-free. Any suggestions on how to fix this would be appreciated.

      mergedata.jpg

        • 1. Re: Merge data creating duplicate listings
          Peter Spier Most Valuable Participant (Moderator)

          Data Merge preview is known to be buggy and to cause all sorts of strange errors (though that isn't necessarily all of the problem here). The best advice is NEVER use it for multiple records/page merges, or if you feel you must, UNDO after you use it, then just merge and trust your layout.

           

          If you've used preview and then gone from there to merge directly, and the merge is not successful, you must make a new template as the one you are using will never work correctly. You can copy/paste in place to create the new template from the old layout.

          • 2. Re: Merge data creating duplicate listings
            wildcoastpub Level 1

            Okay, so  I rebuilt the template document, pasted the data field placeholders into it, merged the data directly and behold they appeared as they should. Weird that a preview would be a bug -- data merge has been in inDesign since at least CS4. They've had a few years to fix it. So thanks Peter, that did the trick.

             

            Now a new problem. In viewing the merged document some fields have overset text, and I've corrected that in the database, but the 'Update content in data fields' option is greyed out. Can the data in the merged document not be updated automatically? If so, how to ungrey the box? When the document was automatically created, the database was not linked, so I had to do that manually. Could that be the problem? Thanks if you can help.

            • 3. Re: Merge data creating duplicate listings
              MW Design Level 4

              The merged document is static...there are no more merge fields in it. Corrections need to be done in the template and a new merge occur, or the merged document needs altered directly.

               

              Mike

              • 4. Re: Merge data creating duplicate listings
                Peter Spier Most Valuable Participant (Moderator)

                The bug was introduced in CS5, I think...

                 

                Update the data only works, as I recall if the fields are on the master page, too.

                • 5. Re: Merge data creating duplicate listings
                  wildcoastpub Level 1

                  Okay, so by moving the fields to the master page, it does indeed retain the ability to update the document from the database. The problem is that removes the multiple records per page option. So I get a 600 page document instead of 150. I can't see a run-around to this; it seems structural: put multiple record templates on the master page and you just create multiple identical records per page. I'm going to ask out loud why Adobe would introduce this function, obviously embrace it over a long period of time and leave it short of the ability to function at a pretty rudimentary level. These are major issues: you can't preview fields without destroying your document, a problem inherited from at least CS4. That's what, a decade to fix it? At least have a pop-up warning: 'Beware! Using this function will ruin your document!' Then you can't update your fields from the database, which is pretty much the point of a database... Unless you jerry-rig the document and it still doesn't function as it should. As a side note, when I went to move a record from a 600 page document to a new home, the program crashed. I just moved us to CC from CS4, and it seems more twitchy and glitch-prone. How is that possible? Just asking out loud after yet another day of dealing with Adobe errors in inDesign and Illustrator.

                  • 6. Re: Merge data creating duplicate listings
                    MW Design Level 4

                    Merge in ID is really not appropriate for catalog work. Never has been, likely never will be. There are commercial plug-ins for ID that do a far better job, as well as far more extensible plug-ins for merging.

                     

                    Mike

                    • 7. Re: Merge data creating duplicate listings
                      wildcoastpub Level 1

                      Well, it would be appropriate for me if the features worked as you'd expect. Mike, can you recommend a plug-in for merging that might address this sort of issue? I'm not in a position to consider EasyCatalog or similar. The software budget went to the upgrade to CC as the Merge Data function seemed adequate for our needs. Ha!

                      • 8. Re: Merge data creating duplicate listings
                        MW Design Level 4

                        Well, they sort of do work as designed...exception being the buggy preview. Data Merge is really designed, I think, for things like labels.

                         

                        The least expensive option that probably can do what you need would be Em Software's InData.

                        http://emsoftware.com/products/emdata/

                         

                        I use Xdata in QuarkXpress. It requires a bit of "programming" in setting up the merge, but it is pretty extensible and powerful as far as merges go. If you can ever swing the cost InCatalog or Teacup's catalog plug-in, though, those would be better options. I'm partial to Em's stuff and so lean that way.

                         

                        Merging using ID's built-in capability is more a one-time, then fiddle around type of affair. Even with the master page option, you can thread all the frames using a script, delete all but the first page/frame, then automatically flow to new pages. I have built catalog sections that way.

                         

                        Another option, also not for the faint of heart, is using XML. Probably a (far) larger learning curve than learning any of the mentioned plug-ins.

                         

                        Mike

                        • 9. Re: Merge data creating duplicate listings
                          Peter Spier Most Valuable Participant (Moderator)

                          I don't consider a lack up update ability that big a deal. It's so simple to run a new merge...

                          • 10. Re: Merge data creating duplicate listings
                            wildcoastpub Level 1

                            Depends on the document. ISSUU - 2015 Kayak Gear Guide by Wild Coast Publishing. This went live unpublished for client proofing requiring two separate sessions, so three reflows of the database. If it worked properly, the clients submit the revised spreadsheets that are dropped into a database. The revised database is then loaded into inDesign. If the update worked as it should, it wouldn't require a single extra piece of work in the inDesign document. It would be: import client spreadsheet into the database, export the database into csv file, update csv into inDesign and update the fields. Five minutes. Now look at that document and you have 60 of 604 records to change and it can't be automated. You're telling me it's no deal reflow that document and build it from scratch again? I don't think so. The problem is this would be SO easy to work properly. Some initial finessing of the headings and bam, done for the year. We could take client updates and reflow it live online in minutes of effort. It should work properly. I paid Adobe to have it work properly. It doesn't work properly. I lost days of productivity trying to overcome the shortfalls and apparently I'm hearing I can't resolve the issue without an expenditure of hundreds potentially thousands of dollars for another system. If that was the case I wouldn't have upgraded to CC. CS4 worked fine and it was paid for. (As an aside, there was an additional error that the links didn't work as they pointed to the spreadsheet rather than the URL. Plus the issuu.com publisher turned all links live, including the database links that pointed to the database, so there were suddenly 10 links or so per record, none of which worked. This version of the document was saved as a PDF without hyperlinks as we decided that no links temporarily was better than 10 that didn't work. So that part remains to be done, another problem to overcome, as the autodetect hyperlinks seems to have a flaw as well when dealing with merged data).  

                            • 11. Re: Merge data creating duplicate listings
                              Peter Spier Most Valuable Participant (Moderator)

                              You update the database in exactly the same way either way. The only difference is the difference in processing time for ID to redo the merge vs. updating the data already merged. For 600 records I would expect the merge to be a matter of only a few minutes.

                              • 12. Re: Merge data creating duplicate listings
                                wildcoastpub Level 1

                                So by merge you're saying use the 'create merged document' command? That gives 600 records in a new document. That's fast but still requires considerable work to finalize it. But I did just think of something.... Creating blank records to fill the spaces used by headings so the headings can be dropped into those slots rather than rebuilding the pages.... Hmmm...