19 Replies Latest reply: Apr 26, 2012 12:34 AM by RDBoon RSS

    InDesign Data Merge Issues...

    RDBoon Community Member

      Hey all, I'm relatively new to using InDesign, I've used it previously for basic artwork layouts etc, and I'm familiar with Illustrator and Photoshop, but InDesign is something I'm still getting to grips with.

       

      I run a mailing company and we have an ongoing requirement for variable data merging. Software designed for variable data work when dealing with PDFs and high res images is incredibly expensive, and so I figured I could try and make use of InDesign's own data merging feature.

       

      Now I'm fully aware of some of the irritating little issues surrounding this tool, such as the way if you've got two blank fields it only removes one of them, and I also know it used to have an issue where it would pull data through from the next record to replace blanks, but I don't see that happening any more.

       

      There are lots of webpages and forums discussing the blank fields problem, but my problem is currently something else.

       

      I've noticed that some of the fields often don't like to display anything other than the header row, the field name.  You drop the field onto the page and create your merge, but when you preview it or publish it whilst most of the fields do as expected and change to show the data, some of the fields do not, they just continue to show <contact> for instance, rather than a name.

       

      I'm at a loss to explain this really, it's not even particularly consistent.

       

      As an example of my problem, I laid out the following address block

       

      <Contact>

      <Add1>

      <Add2>

      <Add3>

      <Add4>

      <Add5>

      <Postcode>

       

      What I get sometimes is this...

       

      <Contact>

      123 Sample Street

      Sample Town

      Sample County

      Sample Postcode

       

      Or...

       

      Mr AB Sample

      123 Sample Street

      Sample Town

      <Add3>

      Sample Postcode

       

      If you add the same field again beneath the one that is playing up, the new field will work, but the one beneath it will then play up, so where in the example above it is <Add3> that isn't merging, if I added another <add3> beneath it, I could almost guarantee that <Add4> would then play up.

       

      I just don't understand it, and it's frustrating me immensely.

       

      Ironically I know a local professional printers who are using InDesign for all their digital variable print runs, without issue, so I'm not sure what I'm doing wrong. They are using Macs whereas I am using a PC with Windows 7 Ultimate x 64bit, but I've had the same problem on other Windows version, so is it a Windows related problem, or are Mac users just not spotting this??

       

      Anyone who has ever experienced something similar and perhaps found a solution, please help!!!!!

       

      Best Regards

      Richard

        • 1. Re: InDesign Data Merge Issues...
          Rik Ramsay Community Member

          It looks like some of your data source is not complete. The missing <add3> are less of a concern that the <contact> as if you mail something out to someone with no name, its obviously spam. You can go through your data source, find the empty <contact> fields and input a default name - Customer/Current Ocuupant etc. Someone on here might know if you can set a default fall back for that kind of thing but I have never seen one.

           

          In terms of your fields showing, you should be able to select 'Remove blank fields' and they will stop showing if there is no data in those fields. See attachment.

           

          placement_options.jpg

          • 2. Re: InDesign Data Merge Issues...
            P Spier CommunityMVP

            Data Merge in CS5 (and very likely CS5.5) has a new bug, which might be what you are seeing, where you unpredictable results in a merged file if you preview the merge first on a multiple records per page merge. In all my testing, it has worked fine if you either do NOT preview, and go straight to merge, or if you Undo the preview step, then merge.

            • 3. Re: InDesign Data Merge Issues...
              RDBoon Community Member

              RikRamsay14 - thank you for your response, the issue regarding blanks isn't really the concern here, I'm aware of this issue, and the remove blank lines works where this is only one blank field, if there's two blank fields then it only seems to remove one. But I can happily work around this by just shifting all my data across. The data source is fine and is not the problem.

               

              Peter Spier - Many thanks for your reply Peter, I thought you might be on to something with this, so I tried opening it from scratch and merging instantly without the use of any previewing, at first I thought it had worked, but when I got further down the merge results I spotted the same problem, field headers appearing where content should be.  These instances are not even where the data cell is blank, it just decides not to display the content, but to display the field header instead, it's inconsistent too so it's really hard to analyse to find a pattern.

               

              My layout incorporates multiple text placeholders containing multiple variable fields.  I am starting to wonder if this could be contributing to the fault. I have tried just creating one text placeholder, and then using the multiple record layout option to display the other three images (it's a postcard four-up on an SRA3 sized page, with four address blocks).  This seems to be less buggy this way, but I have less control with the layout. I'm going to continue to test this theory today, but it could simply be that it can't cope with multiple variable address blocks.

               

              Thanks for all your help guys.

               

              Regards

              Richard

              • 4. Re: InDesign Data Merge Issues...
                P Spier CommunityMVP

                My next thought would be data corruption in the source file, then. Open it in a plain text editor and look for odd characters or the wrong number of fields in a record.

                • 5. Re: InDesign Data Merge Issues...
                  RDBoon Community Member

                  Hi Peter, the data source is fine, I'm using it as test data because I know it's okay. We used it just last week to do another mailer and had no issues.  I've also tried a different data source, both in CSV, and Tab delimited version, both give me the same error. I've also tried a completely different computer, running a 32bit OS instead of 64. With no luck unfortunately! I will keep trying!

                  • 6. Re: InDesign Data Merge Issues...
                    P Spier CommunityMVP

                    Have you tried re-saving the data file from a plain text editor? And rebuilding the template?

                    • 7. Re: InDesign Data Merge Issues...
                      RDBoon Community Member

                      No, but I will give that a go shortly. Thank you.

                      • 8. Re: InDesign Data Merge Issues...
                        P Spier CommunityMVP

                        Fpr what it's worth, which is not much, I saw a data file back in CS4 that behaved in a manner similar to what you describe, but worked OK in CS3, though I was never able to reproduce the problem with any other data source or locate what the issue was.

                        • 9. Re: InDesign Data Merge Issues...
                          RDBoon Community Member

                          I'm actually using CS4 at present. Wonder if the upgrade would help?!

                          • 10. Re: InDesign Data Merge Issues...
                            P Spier CommunityMVP

                            You could download the trial of CS5.5....

                             

                            If you have CS3, you might try the merge there, too.

                            • 11. Re: InDesign Data Merge Issues...
                              cdflash Community Member

                              I use InDesign CS5.5 (Mac) after upgrading directly from CS3, and found few issues with the Data Merge, and I really push DM to do things it wasn't intended to do! Did have problems during CS5 as Peter reported in an earlier post, but can't fault it lately.

                               

                              Haven't experienced the error that the OP is experiencing though... ever.

                               

                              What I will say is that good data = good merges. Data that contains soft returns or other characters which shouldn't be there often ruin mail merges, and this has happened a few times.

                              • 12. Re: InDesign Data Merge Issues...
                                RDBoon Community Member

                                My own background is heavily data orientated, from cleansing and management to database development and maintenance, I'm no designer, so I come into this from a data driven perspective. That's why I know my data is good, that and I've had the same problem with multiple data sources now.  I'm going to try a few more things and then look at the trial for 5.5

                                • 13. Re: InDesign Data Merge Issues...
                                  RDBoon Community Member

                                  The plot thickens. There is a pattern, but it's got harder to decipher! Turns out it's also pulling information from the next record to replace names or address lines that it doesn't like in the previous record. So it could be a data side problem I suppose but there seems nothing within the data that could cause some of this information to not display.  What it's doing is deciding it doesn't like a Contact Name, replacing it with the Contact Name in the next record, and then eventually when it gets to the last record and has no name left because it's one out, it puts the field header!

                                   

                                  It does this on address lines too, so you end up with a mix of wrongly assigned names and address lines, until you get to the last records where you get the field headers because there's no data left to merge!  Absolutely bizarre. But I have known this to happen before and it is a known issue that I've spotted other people having online.

                                   

                                  I could understand it if it was replacing empty fields with information from the next record, but it's not, it's actually missing out cells completely and then moving the data from the next field up to replace it.

                                   

                                  There's more to it than that too, but it's just so complex to explain. Sometimes it doesn't like a name, so it replaces it with a name from the next record, but doesn't move the information up, just duplicates the name up to the next record. So it's then right again on the next record. But then it will have a problem with an address line and the whole process starts again!!! I'm completely at a loss now!  And I can see nothing in the data to explain this. It's a tiny data file with a field for each line of address and absolutely no other characters or symbols and no returns.

                                   

                                  Lost.

                                  • 14. Re: InDesign Data Merge Issues...
                                    RDBoon Community Member

                                    Okay, after much testing and pulling hair out, I've clarified that it's not the data that is the issue, it's most definitely a bug within InDesign itself.  If I use mulitple record layout to show four records on the one page then it's actually fine (albeit still not removing more than one blank field per record), but there's no random displaying of field headers (which only happens on the first and last records?!?)

                                     

                                    I don't really want to use multiple record layout as I want more control myself as to how it is all laid out, but is the only option for me at present, the moment I try and have multiple placeholders referencing multiple fields it fails.

                                     

                                    Thanks for all your help people, I will upgrade to 5 and see if that helps but in the mean time I will have to try and use multiple record layout!

                                     

                                    Regards

                                    Richard

                                    • 15. Re: InDesign Data Merge Issues...
                                      P Spier CommunityMVP

                                      RDBoon wrote:

                                       

                                      I don't really want to use multiple record layout as I want more control myself as to how it is all laid out, but is the only option for me at present, the moment I try and have multiple placeholders referencing multiple fields it fails.

                                      OK, now I'm confused. Data Merge has two options, one set of placeholders and you do a multiple records per page merge, or duplicated sets of placeholders and you duplicate the same record multiple times on the same page.

                                       

                                      Just what do you mean by "multiple placeholders referencing multiple fields"?

                                      • 16. Re: InDesign Data Merge Issues...
                                        RDBoon Community Member

                                        Hi Peter, sorry for the confusion, basically my data source has multiple names and addresses in one record (across several fields), this is when we have family units which are stored in one line etc but have to be displayed at different places on the artwork.  Or sometimes it's where we want to print something four-up on a sheet to four different people without having to get InDesign to do the multiple setup for us.

                                         

                                        Anyway, I was about to come on here anyway because I've literally spent the last two hours upgrading to CS5.5 and testing it, and guess what? - Absolutely no problems whatsoever!!!!

                                         

                                        Even the inability to successfully remove two blank fields in a row seems to have been fixed, so I'm convinced (obviously) that it was a problem within CS4.  I had never used CS3, 4 was my entry to this world, so I can't speak for previous versions, or for 5 for that matter, but 5.5 is perfect, and as long as it stay this way it should be a long and prosperous relationship!!!

                                         

                                        Thank you for all your help, I still don't know why InDesign CS4 had this annoying problem, but it didn't like multiple place holders covering lots of fields, it was okay if it duplicated everything itself but didn't like lots of fields and lots of placeholders. CS5.5 has no such concerns, it just gets on with it :-)

                                         

                                        Very happy now!! It's been a stressful week.

                                         

                                        Many thanks all.

                                        • 17. Re: InDesign Data Merge Issues...
                                          P Spier CommunityMVP

                                          And now it's Friday. Bonus!

                                          • 18. Re: InDesign Data Merge Issues...
                                            Mr. Adam Carroll Community Member

                                            Hi Everyone,

                                             

                                            I encountered this problem, or one eerily similar, myself today (merge fields jumbled up for seemingly no reason), and found the problem to be my datasheet.

                                             

                                            My datasheet was full of information for business cards (Name, Mobile, Address, etc) but after exporting to CSV and performing a data merge in CS5.5 the fields were jumbled up seemingly at random.

                                             

                                            The issue was that my address field used line breaks (alt + enter in Excel) to separate street addresses from cities.

                                             

                                            By removing the line breaks and setting each element of the address into a seperate column (Excel) and corresponding merge field (InDesign) the issue was resolved.

                                             

                                            My guess is that either InDesign's Data Merge does not handle line breaks very well, or when converted to CSV the data formatting can't handle line breaks.

                                             

                                            I hope this helps!

                                            • 19. Re: InDesign Data Merge Issues...
                                              RDBoon Community Member

                                              Thanks for that Adam, I'm pleased you managed to resolve your issue by adjusting your data format. Mine was not quite that straight forward, there were no line feeds/breaks, carriage returns or anything such like, we have to remove all such characters from our data before we process it ready for print, it's standard practise for us.

                                               

                                              My problem seemed more random, some address lines just would not be merged through and would be replaced by the address line in the next record, and then the last few records would have header rows in because there would be no address data left!

                                               

                                              Thankfully switching to CS5.5 completely fixed this problem. We've been using CS5.5 now for a few months without a single problem.