diamond question mark character set

Questions about the MySQL, PostgreSQL, and most other databases, as well as using it with PHP can be asked here.

Moderator: General Moderators

Post Reply
User avatar
kendall
Forum Regular
Posts: 852
Joined: Tue Jul 30, 2002 10:21 am
Location: Trinidad, West Indies
Contact:

diamond question mark character set

Post by kendall »

Hey guys,

I have a PHP/ MYSQL driven content website in which I have alot of latin and euorpean characters in the MySQL Tables....

Now in PHPMYADMIN the characters show up fine.....But on my website My characters are converted into <?> in my web pages.

i have tried using utf-8 and iso... but to no avail

I tried re-importing an sql file with the data to no avail. I'm looking at the characterset encoding of the columns and some are latin_swedish_ci and others are utf8_general_ci. Whats the appropriate encoding to use for such non-english characters e.g. é and so fort...

for an e.g.

http://cre-ole.com/get_restaurants.php? ... =10&page=3
User avatar
Apollo
Forum Regular
Posts: 794
Joined: Wed Apr 30, 2008 2:34 am

Re: diamond question mark character set

Post by Apollo »

kendall wrote:Whats the appropriate encoding to use for such non-english characters e.g. é and so fort...
Any encoding that can represent those chars. For typical latin characters such as é and ä etc, both iso-8859-1 and UTF-8 would do fine (as well as several other encodings) but obviously, it only works as long as you're actually outputting content encoded the way you say it's encoded.

Same for your database: setting the right collation on a SQL table is one thing, of course you have to make sure that the text you're storing is actually encoded that way.
Looking at that page's html, the content is in fact UTF-8 encoded, but they represent some Chinese unicode characters (which you probably can't display, hence the question marks). Seems like you over-converted your content, or converted incorrectly.

Don't randomly convert data or change HTML encoding or SQL collations if you don't know what you're doing. If you still have the original text, you should check what original encoding it is in, convert it to the encoding you prefer and set the collation and HTML encoding accordingly.
User avatar
kendall
Forum Regular
Posts: 852
Joined: Tue Jul 30, 2002 10:21 am
Location: Trinidad, West Indies
Contact:

Re: diamond question mark character set

Post by kendall »

Apollo wrote: Don't randomly convert data or change HTML encoding or SQL collations if you don't know what you're doing. If you still have the original text, you should check what original encoding it is in, convert it to the encoding you prefer and set the collation and HTML encoding accordingly.
Well I don't have the "original text" per say...just what was extracted from a database...the SQL statements all have the direct character itself i.e. it isnt html encoded. In phpmyadmin it shows up ok so it seems that phpmyadmin is converting it accordingly and php is browser based. So then what do I do? can I re-convert it? should I try re-importing it? lets take the "name" in the black all i'm doing is "echoing"
vikram_u
Forum Newbie
Posts: 5
Joined: Fri Nov 13, 2009 11:59 am

Re: diamond question mark character set

Post by vikram_u »

Have a similar issue wherein the Data i.e (Gagné, Michel) in DB is in UTF8, Table/Schema and are charset: UTF-8.



In Firefox and Chrome the data is seen with Diamond[] and in IE, the View Source indicates a missing character. I still see the Black Diamond. Any suggestions..?

Php.ini and httpd.conf charset are disabled as per this thread http://blog.salientdigital.com/2009/06/ ... omment-110

i.e phpinfo() displays:
default_charset no value
Page encoding is set to utf8.
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
CharSet.PNG
CharSet.PNG (34.39 KiB) Viewed 3623 times


Thanks in advance.
Last edited by vikram_u on Fri Feb 26, 2010 8:49 am, edited 1 time in total.
User avatar
kendall
Forum Regular
Posts: 852
Joined: Tue Jul 30, 2002 10:21 am
Location: Trinidad, West Indies
Contact:

Re: diamond question mark character set

Post by kendall »

I think i had to go fiddle with the table's character set initially. Like stated from previous post this is something that should have been set long before the data as to avoid things like these. Its not a perfect solutions but what you would need to do is play with the table charact set the mysql_connect charct set and the php charct set a bit to see the results...i dont recommend this though...if there is some way you can re write the data then it suggest you try that....sorry if im not much help
vikram_u
Forum Newbie
Posts: 5
Joined: Fri Nov 13, 2009 11:59 am

Re: diamond question mark character set

Post by vikram_u »

Key here is that the table data is in UTF8, encoding is in UTF8, PHP and Apache have no charset modifications.

The HTML displaying this is encoded as utf8. Still all browsers fail to display. Amazes me?.
User avatar
Apollo
Forum Regular
Posts: 794
Joined: Wed Apr 30, 2008 2:34 am

Re: diamond question mark character set

Post by Apollo »

vikram_u wrote:Key here is that the table data is in UTF8, encoding is in UTF8, PHP and Apache have no charset modifications.

The HTML displaying this is encoded as utf8. Still all browsers fail to display. Amazes me?.
I think the data in your table is not really UTF8. Sure, its collation may be UTF8, but if you put ansi bytes in there, for example because you once filled the table using an online form which had an iso-8859-1 content HTML header, you've still been feeding the database with wrong data.

The reason phpmyadmin may still display it correct is because it messes up with character encodings at several places, and perhaps it's erratically showing the ansi data (which should have been utf-8) as-is, i.e. ansi.
Similarly, you could see something like "Gagné, Michel" in phpmyadmin, in which case the table actually holds the correct UTF-8 encoding of "Gagné, Michel" but interpreted/displayed as ansi by phpmyadmin.
Post Reply