Page 1 of 1
how do I get rid of funky characters in db table
Posted: Thu Jul 12, 2007 3:43 pm
by Luke
I have a database table full of products. The client copy & pasted from word into miva (miva sucks) and miva doesn't clean the input, so I have a bunch of funky characters in my data. Could I clean this data by converting my database table to utf-8, or would that not even help at all? If it would, are there any adverse effects I should be aware of.
If this is totally silly and of course it wouldn't work, well... what would?
Posted: Thu Jul 12, 2007 5:21 pm
by ReverendDexter
I know this might be considered kinda "low-rent", but you could always do something along the lines of a white-list search & replace under Vi/Vim (or another text editor that does regex s&r).
Maybe the following?
Code: Select all
:%s/[^A-Za-z0-9!@$%()+=-><\[\]\\\/]//g
I mean, add the appropriate symbols (and escaping) as necessary...
EDIT: I noticed a glaring error in that regex, there's probably less glaring errors left.
Posted: Thu Jul 12, 2007 5:40 pm
by pickle
If you've got a linux box, try using the 'recode' program. It'll open a file & try to re-interpret it using a character set you can specify. It might clean things up a bit.