RH UTF-8 Compatibility in AIX

[xsakita]

Hi,
I'm working in RoboHelp 7.02 and compiling WebHelp for multiple platforms. When testing in AIX, the product deploy scripts are hitting on many of the files stating that they aren't UTF-8 compatible (and stopping the deploy). If we bypass UTF-8, we can get past it. The QA engineer also opened many files and saved them in AIX, and those files now make it through the deploy. My HTML files check out with the char set to UTF-8. I've updated my Windows box to add the supplemental languages and regenerated, but this didn't help. I'm not sure what to try next. This wasn't a problem for our Solaris or Windows environments, and AIX has to be UTF-8 for some of our 3rd party tools. QA developed a workaround by using the default setting during deploy, then changing to UTF-8 afterward. I haven't yet checked to see what this might do to any special characters in the help. And I'd rather fix the source. Anyone have any ideas on any settings on Windows (XP SP3) or RoboHelp to get the generated files set and recognizable on AIX?
Thanks,
Andrea

[Peter Grainge]

Welcome to the forum.

Are the topics created using the RH editor or a third party tool?

Provided they have been edited using the RH editor, take one topic from the output and the "same" file from the source and put them in folders named source and output in a zip file. Send them to me and I will get someone to look at them to see if the problem can be identified.

Send the zip via my site.

[xsakita]

Hi Peter,
Thanks for responding. I have a little more information from my dev and QA teams. I am using the RH WSIWG editor for the topics. The problem doesn't seem to be with the topic files. They all seem to be making it through. The problem seems to be with the generated files such as the .js and .jar files. Seems that this deploy script is checking each file it deploys to ensure that it is UTF-8 compliant. These files are stopping the deploy. For example, ehlpdhtm.js is one of the files. If the QA engineer opens it in AIX and saves it, the file makes it through. This is the error for VI if you try to open one of the files with the problem:
"ex: 0602-169 Incomplete or invalid multibyte character encountered, conversion failed.ex: 0602-169 Incomplete or invalid multibyte character encountered, conversion failed.ex:
0602-169 Incomplete or invalid multibyte character encountered, conversion failed.ex: 0602-169 Incomplete or invalid multibyte character encountered, conversion failed. 4334 lines, 122651 characters

It seems as if these files are not UTF-8 compatible. I've checked everywhere I can think of in RH to find someway to ensure the output is UTF-8. The topic files state that they are, but these generated files don't show any headings to set UTF-8. I'm unsure as to how to do this. I'm great at debugging FrameMaker, but I'm still getting the hang of RoboHelp. One more point, the products using our installer work without error on our help files, so I suspect the installer handles the UTF-8 or ignores the help files entirely. It's just the manual deploy script that is failing on these files.

Do you want me to send you any of the generated files that are causing the problems?
Thanks!
Andrea

[Peter Grainge]

Andrea

I will refer your reply so send nothing for now.

Meantime, get your QA guy to look at Item 22 at
http://www.grainge.org/pages/authoring/rh7/using_rh7.htm

I know the symptom is different but your comment about the QA guy opening the file and then it gets through seems to have something in common.

[xsakita]

Peter,
I actually checked out your site for 22 and 10 when searching other responses. 10 didn't help and I couldn't see an answer with 22, but I will talk to my QA team about it to see if they can give me something new to try. I will reply again if they come up with an answer.
Thanks,
Andrea

[xsakita]

Hi,
I did just check with the QA team for issue 22 from your site. The deploy script is validating each file, probably looking for the UTF-8 BOM your site mentions. The generated files that had problems don't have the BOM, but are working in a UTF-8 environment. I'm tempted to have the development team rewrite their script for now to ignore the help file directory, but this isn't really the ideal solution. I wonder if Adobe has a bug filed on this already for adding this BOM to the generated files.
Andrea

[Peter Grainge]

If it is the same cause, they are aware of it. If not, I have asked someone to take another look at the thread so we will know from that.

[xsakita]

Thanks Peter. I'll keep checking on this. It certainly seems to fit the cause. I need dev to confirm what their deploy script is doing to validate, but am waiting to talk to them.

[xsakita]

Our developers are using an Apache Ant script to validate the files. They say it's not just looking for the BOM, but also the content. There seems to be some problem with the trademark and copyright symbols in the files. ehlpdhtm.js has several symbols in the commented text, but this is apparently still looked at by the ant script.

[xsakita]

We're also having problems with the jar file included in the generated files. Seems this is corrupted and WebSphere can't handle it. I found the following thread from 2004, but can't determine if Adobe ever fixed this problem:

http://www.adobe.com/cfusion/webforums/forum/messageview.cfm?forumid=65&catid=447&threadid=1379878&e...

Peter,
Have you heard anything about the copyright symbols as a source of the other problem (from my last message)? I'm wondering if editing the source file to remove the comments might help.

Thanks,
Andrea

[Peter Grainge]

I don't think anyone else has posted with issues with webhelp.jar and I don't recall anyone else posting about symbols.

[xsakita]

The link I included in the last message points to a problem with the jar file in RH 5. I need to know that the corrupted file was fixed in RH 7. To complicate this mess, the last version of this product help was generated in the older help version. The dev team's build didn't get the new files I generated for RH 7 in all cases. This has brought up quite a bit of confusion. The .jar file is corrupt according to WebSphere, so I want to check on whether or not this was fixed in RH 7 so I can go back to the dev team and make sure they're really building with the latest files.

The symbols issue seems to be a problem in the old and new builds tested by QA. That does seem to be what is stopping the deploy. The copyright symbols are there from the old eHelp comments.

[Peter Grainge]

Yes I understood that. When I said "I don't think anyone else..." I meant in the sense of anyone other than those posters. Only Adobe can say if it was fixed. I have asked someone to look at this but I am sure you will realise it is not a five minute task. The person will be doing this as a favour as when a problem is outside the scope of forum supporters, who are authors like you and me, the correct route is for you to go to Support. As such they have to fit it in as best they can. If this is not moving fast enough for you, then I have to point you to Support.

I have seen reports where a virus checking program says that a file in an application has a virus. Then when the virus company investigates they find it is not a fault with their software and the application containing the file gets patched. I am saying that WebSphere is the problem, just that as others are not having these issues, that has to be considered.

Please bear with us, but if you cannot, then you will have to take it up with Support. I suspect on this they will end up going where I have gone.

[xsakita]

Peter,
No problem. My developers are a bit impatient because we were trying to get out a pre-release version yesterday. I think I have them calmed down now and we can wait for the process for a bit. For the symbols and all, I was just passing on info and questions as they came up. I'd love to blame WebSphere for part of this. WebLogic doesn't have this issue. I'm putting new files in the build again to see what QA comes up with on the next round. Any help from this forum would be great. If we can't solve it here before our GA in Dec, I'll definitely bring it to Adobe support.
Thanks again,
Andrea

[Peter Grainge]

I have been advised that it is just the html topics that need to be
UTF-8 encoded. If there is some other reason, beyond satisfying how WebSphere works, then please ask your QA people to explain that so that it can be considered for a future release.

As to the comments, I am told these can be safely removed.

[xsakita]

Thanks Peter. I'll get back to you if my QA team has any new/different information.