6/13/2005

MySQL and Python and Unicode

I have just wasted waaaay to much time (a day) going through the inner workings of unicode, mysql, python (and twisted) - to get something working I assumed would be simple.
I'm going to document it here so I never have to go through it alll again and hopefully help a few others out along the way. Python's unicode support is quite good. For an excellent intro check out End to end unicode web applications in Python.
It turns out my issue was with MySQL. The trick is that everything everywhere should be set to UTF8. The best way to test this is run:
SHOW VARIABLES LIKE 'char%';
which should result in something like:
character_set_client | utf8
character_set_connection | utf8
character_set_database | latin1
character_set_results | utf8
character_set_server | utf8
character_set_system | utf8

(having database set to latin1 seemed to be fine, but would be nice if it was utf8)

How can such a thing be achieved? I'm sure there's some settings that can be placed in my.cnf and/or startup to do the above, but I haven't found them yet (and #mysql wasn't a great help either)
My solution for now is to perform the following 3 queries when establishing a connection:
SET NAMES utf8;
SET CHARACTER SET utf8;
SET character_set_connection=utf8;

It is important to do this for all operations, inserts/selects/updates etc. I'll post a better solution when I track one down, but this works for the time being.

EDIT: With Python when establishing a database connection add the following flag: init_command='SET NAMES utf8'.
In addition set the following in MySQL's my.cnf:
default-character-set = utf8

That should fix pretty much everything to utf8 for nice i8n support :)

3 Comments:

Anonymous Anonymous said...

I have been to your site and I too am working very hard at adsense cash cash cash gogle google luckysign.googleearn.hop.clickbank.net machine machine power to increase my revenue. I am also looking into many NEW ways to utilize the design to further direct people to follow my ads.
adsense cash cash cash gogle google luckysign.googleearn.hop.clickbank.net machine machine power

9:34 AM  
Blogger blaze said...

Great info about adds adsense free google googleadsensesearch.com I have a lage interest also and have a site about adds adsense free google googleadsensesearch.com please feel free to check it out.

4:40 PM  
Anonymous hp said...

hmm... lots of spam here!

anyways, the little rant on getting mysql to default to utf-8 was very useful.

thanks a bunch!

5:38 PM  

Post a Comment

<< Home