Unicode (international) characters in cfpdfform with LiveCycle PDFs

[jtchivers]

Hi all

These are the details of my environment:

ColdFusion version: 8.0.1
Update level: C:/ColdFusion8/lib/updates/chf8010004.jar
Server OS: Windows Vista 64
Web server: Apache

I'm using cfpdfform in ColdFusion 8 to read a Livecycle 8.2.1 pdf form straight into the browser window, enter text into a couple of fields and then submit the form back to the same CF page and use the submitted XML to populate the same form.

So, the relevant bit of my code is as follows:

<cfif IsDefined("PDF")>
    <cfpdfform action="read" source="#PDF.content#" xmldata="myXMLData" />
    <cfset myXMLData = XMLParse(myXMLData)>
    <cfpdfform action="populate" source="c:\temp\myPDFForm.pdf" xmldata="#myXMLData#" overwritedata="true" />
<cfelse>
    <cfpdfform action="populate" source="c:\temp\myPDFForm.pdf">
        <cfpdfsubform name="topmostSubform">
            <cfpdfformparam name="TextField1" value="Hello! The time is #TimeFormat(Now(), 'HH:mm:ss')#" />
        </cfpdfsubform>
    </cfpdfform>       
</cfif>

This all works just fine, except that any multibyte unicode characters (e.g. Chinese) I enter into any text fields are lost (turned to question marks) when the form is populated from the submitted XML, even though I can see that the Unicode characters remain intact if I do a cfdump of the myXMLData variable on page submission.

I ordinarily have no problems with Unicode. I can write unicode characters in multiple foreign character sets to database and retrieve them just fine. I have site pages in several international character sets, and, as I have stated, the text I enter is displayed correctly in the correct foreign characters when I view a cfdump of the submitted form, so this appears to be specifically related to the PDF form populate side of things (or a setting in LiveCycle).

I have an inkling that the cause is that the LiveCycle form has its  text fields set to allow rich text and has a specific font set against  the fields, so I'm wondering whether the foreign character font, which  displays fine when I enter it into the form, is being lost (overwritten by the LiveCycle field font) when read  back into the form. Any ideas as to how to overcome this problem would  be gratefully accepted.

[tooMuchTrouble]

On 11/11/2010 12:01 AM, jtchivers said:

Chinese) I enter into any text fields are lost (turned to question marks)

question marks (?) indicate a serious encoding error (like garbaged data). not

really a LC expert but i can offer a few hints:

- didn't see any cfprocessingdirective, tried that?

- can you writeout the unicode data to PDF ok using cfdocument?

- reading in the new unicode data from an XML file & maybe forgetting the

"charset" option?

I have an inkling that the cause is that the LiveCycle form has its text

fields set to allow rich text and has a specific font set against the

fields, so I'm wondering whether the foreign character font, which displays

fine when I enter it into the form, is being lost (overwritten by the

LiveCycle field font) when read back into the form. Any ideas as to how to

overcome this problem would be gratefully accepted.

i want to say not likely, as simple font/rendering errors are usually shown as

empty squares.

[jtchivers]

Hi Paul

Thanks for your suggestions.

I tried cfprocessingdirective, but it made no difference, and since the Chinese characters were displayed correctly in the cfdump, I thought it wouldn't be that (but I wasn't sure).

I dumped the submitted PDF form XML to cfdocument and all the Chinese characters were there and displayed correctly.

<cfdocument format="PDF">
     <cfdump var="#myXMLData#" />
</cfdocument>

- reading in the new unicode data from an XML file & maybe forgetting the

"charset" option?

I'm reading the XML in directly through the cfpdfform tag, so there is no charset option. Since the Chinese characters are displayed correctly on page or to a cfdocument, I don't believe the problem is on the read, but on the populate action.

Out of interest, I tried copying the Chinese characters directly into the PDF form field on LiveCycle and saving the form. When the form was displayed in the browser (using cfpdfform and action populate), the Chinese characters were intact.

This all points strongly towards the issue being the populate action on the cfpdfform tag, but there appears to be very little I can do about it (ColdFusion's power is that I can read a PDF form and write to one using the cfpdfform tag, but it does mean that there is limited scope for errors, which is good on the one hand and bad on the other - if it's a bug, I'm a bit stuck).

It's not really a LiveCycle issue from what I can see, as LiveCycle is retaining the Chinese characters in a form when they're already present.

Thanks for your help and suggestions anyway, Paul.

[tooMuchTrouble]

let me see if i can get the attention of somebody who knows more about cfpdfForm

than i do. its for sure an encoding error, just need to see where we can affect it.

[jtchivers]

Okay, I think I got to the bottom of this. The following full page code works:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>cfpdfform Post and Populate Example</title>
</head>

<body>

<cfoutput>

<cfif IsDefined("PDF")>
    <cfpdfform action="read" source="#PDF.content#" xmldata="myXMLData" result="data" />
    <cfpdfform action="populate" source="c:\temp\myLiveCycleForm.pdf" xmldata="#myXMLData#" />
<cfelse>
    <cfpdfform action="populate" source="c:\temp\myLiveCycleForm.pdf" />
</cfif>

</cfoutput>

</body>
</html>

The key to stopping the Chinese characters from breaking was to remove the overwritedata attribute from the populate tag on the action page. In other words, changing

<cfpdfform action="populate" source="c:\temp\myLiveCycleForm.pdf" xmldata=""#myXMLData#" overwriteData="yes" />

to

<cfpdfform action="populate" source="c:\temp\myLiveCycleForm.pdf" xmldata="#myXMLData#" />

fixed the problem.

As an interim measure, I wrote out the submitted PDF form XML to a file and took a look at what was happening and used the XML file as the basis for populating the PDF form, like this:

<cfif IsDefined("PDF")>
     <cfpdfform action="read" source="#PDF.content#" xmldata="myXMLData" result="data" />
    <cffile action="write" file="c:\temp\myXMLData.xml" output="#myXMLData#" charset="utf-8" nameconflict="overwrite">
    <cfpdfform action="populate" source="c:\temp\myLiveCycleForm.pdf" xmldata="c:\temp\myXMLData.xml" />
<cfelse>
     <cfpdfform action="populate" source="c:\temp\myLiveCycleForm.pdf" />
</cfif>

As well as causing the Chinese characters to break, the overwritedata attribute when populating the form appears to strip out any rich text  formatting (CSS) from form fields.

Dropping the overwritedata attribute preserved the Chinese characters, regardless of whether the PDF form is populated directly from the posted PDF XML (as in the full example above) or via an XML file.

On a separate issue (one I'll post to the LiveCycle forum but I'm mentioning here for other people's reference), any XML which contains in-line CSS formatiing (e.g. individual words in bold text) using the span tag causes a ColdFusion error when used to populate the PDF form.

[tooMuchTrouble]

good catch. maybe raise a bug:

http://cfbugs.adobe.com/cfbugreport/flexbugui/cfbugtracker/main.html