how to handle weird characters in form fields (like bullet points from Word docs)

[Nick Way]

hi there

lots of my users copy and paste from MS Word documents into a text area

my app takes that form.textarea data and inserts it in a MYSQL db.

when we output that cell from the db, we often find there are ? [question marks] where there once were special characters in the Word doc text.

it seems that bullet points in word docs are the worst offenders.

each time i spot a new one i try to find out the chr() number for it and make a substitiution before the text is stored in the db, but its a bit hit and miss (relies on me spotting it) and its v time consuming

ive tried using functions safetext and demoronize - but they dont seem to do what i need

im sure im not the first person to run into this - any suggestions anyone?

thanks very much indeed for any help you can give.

kind regards

Nick

[darbus23]

depending on what you want to do with the text you could always use the <cftextarea> tag with the richtext tag set to "yes". This will give your users a WYSIWYG editor (it's actually the FCKeditor) for the text box and when someone tries to paste Word content it asks them if they want to clean it up before pasting it into the textarea.

[Nick Way]

thanks darbus - i'll check that out - although i suspect that'll require javascript support and our site is supposed to work even if JS is disabled - so ideally i do need to find a server side solution if poss..

thanks v much for your suggestion tho - i'll deff look into that aswell - cheers

[Newsgroup_User]

happysailingdude wrote:
> when we output that cell from the db, we often find there are ? [question
> marks] where there once were special characters in the Word doc text.

because your db's encoding doesn't support ms's latin-1 superset. if you want to
preserve these chars, change it to unicode. cf's default encoding (since version
6) is utf-8 so it's ready.

[Newsgroup_User]

iirc, there is a udf over at cflib.org that will clean up all these high
(or is it low?) ASCII characters for you...

Azadi Saryev
Sabai-dee.com
http://www.sabai-dee.com/

[Nick Way]

Thanks Paul and Azadi

Paul - when you say "change it to unicode" do you mean the page or a setting in the db? forgive me if i'm being stupid but this is all new territory to me. Thank you ever so much for your help - does sound logical what you say :)

Azadi - thanks very much for this (I'm pressuming that there's a charge for this udf) so i'll try to implement Paul's solution and if I can't do that I'll buy the the udf - thanks very much indeed for your help, really is much apprecaited.

kind regards

Nick

[Newsgroup_User]

all udfs at cflib.org are totally free

Azadi Saryev
Sabai-dee.com
http://www.sabai-dee.com/

[Newsgroup_User]

happysailingdude wrote:
> Paul - when you say "change it to unicode" do you mean the page or a setting

cf already defaults to unicode. change the db & use the JDBC driver that comes w/cf.

i'm not a mysql expert but i think it's just

SET NAMES utf8;
SET CHARACTER SET utf8;

BUT make sure you have everything backed up 1st. or even better read the mysql
manual on unicode.