In Golgota

While trying to use MySQL 4.1 and its new utf-8 features, I ran into character encodings problems.
Even though the html page encoding was correctly set to utf-8 and data in the database was correctly utf-8 encoded data, the content displayed in my browser was completely wrong: non ascii characters were wrongly converted to 2 chars instead af one, revealing that MySQL was serving latin-1 instead of utf-8 as it should.
Trying to fix that by setting every default character set to utf-8 in my.conf, the MySQL configuration file, or in the database, didn't solve the problem. Changing collations, etc. didn't do it neither.
By looking at phpMyAdmin's way to solve the problem, I found this query right after they make a connection to the database:

SET NAMES 'utf8'; 

This statement set the character encoding for the next operations. This is a design flaw in MySQL,  they used a hack instead of doing things properly from the beginning. What happens if you use a database wrapper like PEAR::DB or whatever ? You will have to test beforehand if the db in use is MySQL if you need utf-8. Sounds silly ?
So the only way I found to avoid doing $db->query("SET NAMES 'utf8'") everytime I have to use MySQL 4.1, was to add this line to my.conf under the [mysqld] section:

init_connect="SET NAMES utf8" 

This means that everytime MySQL opens a connection, it sends SET NAMES utf8. But as the init_connect directive is not taken into account for user root (argument given is that you won't be able to fix problems under root account if your init_connect directive is wrong... not taking into account that if you are able to change my.conf, you can fix them anyway), you will have to use another account to test your changes.


Lots of time wasted.