6.0.0-git
2019-03-24

[#8777] $conf['sql']['charset'] is ignored
Summary $conf['sql']['charset'] is ignored
Queue Horde Base
Queue Version 3.3.5
Type Bug
State Not A Bug
Priority 1. Low
Owners
Requester e.cerny (at) seznam (dot) cz
Created 2009-12-15 (3386 days ago)
Due
Updated 2010-02-07 (3332 days ago)
Assigned
Resolved 2009-12-15 (3386 days ago)
Milestone
Patch No

History
2010-02-07 06:22:27 lfbm (dot) andamentos (at) gmail (dot) com Comment #8 Reply to this comment
I'm telling you that the code that did this right before is doing it 
wrong now because it relays on PEAR functionality that isn't there 
anymore.

Understand ?

And this make this problem a Horde BUG!
Hello:

I think Horde is doing the work just fine. You can check it by 
manually echoing the result of, for example, convertToDriver method 
from kronolith/lib/Driver/sql.php. You'll see that the text is 
converted just fine.

What seems to be happening wrong is, for some reason I don't know, 
mysql connection always defaults it's charset to 'latin1' (even if the 
charset is set to utf8 in my.cnf in charset_set_server and 
default_charset_set) and then the text with "special" characters, 
which was converted fine at the php layer, is wrongly recorded into 
the database as 'latin1', and thus the problem of unusable data saved.

I managed to solve this issue by adding:

[mysqld]
init_connect="SET NAMES 'utf8'"

and restarting mysqld. Now the text is just the same on horde and on 
the db, which makes it good to be used from other tools (import/export 
etc.).

Also, the mysql_set_charset function only works for PHP 5 >= 5.2.3. So 
the workaround of editing PEAR's DB may no apply to all installs.

See if the init_connect option in my.cnf works for you. Hope it helps.
2009-12-15 23:04:17 e (dot) cerny (at) seznam (dot) cz Comment #7 Reply to this comment
1. This code is in the PEAR DB package which has nothing to do with Horde.
2. The conversion is done to the charset that you configured in your 
database settings. If this doesn't match the charset that your 
database is actually using, that's your fault.
No it's you fault because you doesn't set the charset so it's use the 
unchangeable(so far) charset...

The PEAR code we tallk about has changed and you didn't do the 
apropriet change of your code.

PEAR does not set character set on mysql connect anymore (because php 
does not set character set on mysql database connect) so must you...

It's not about what character set my database is using because it's 
about what character set is using the client side. The mysql module in 
php which is default latin1 (UNCHANGEABLE DEFAULT).
It's on application to change it.
I'm telling you that the code that did this right before is doing it 
wrong now because it relays on PEAR functionality that isn't there 
anymore.

Understand ?

And this make this problem a Horde BUG!
2009-12-15 21:50:05 Jan Schneider Comment #6 Reply to this comment
1. This code is in the PEAR DB package which has nothing to do with Horde.
2. The conversion is done to the charset that you configured in your 
database settings. If this doesn't match the charset that your 
database is actually using, that's your fault.
2009-12-15 20:30:02 e (dot) cerny (at) seznam (dot) cz Comment #5 Reply to this comment
Doesn't matter because we convert the data to the correct charset anyway.
I thought about it and I can say NO YOU DO NOT CONVERT ANYTHING!
The data in database are stored as 8bit representation of multi-byte encoding.
If there was any conversion from mysql  default encoding (latin1, 
which is not able to hold multi-byte characters) to some other 
encoding. It would not obviously work for multi-byte characters.

Horde simply stores utf-8 (if sql-charset is utf8) byte stream in 
mysql as latin1
and the same way retrieves it back.

This bug in combination with php inability to change 
default-character-set for mysql. May be considered not beeing a bug, 
but it is one.
And when php becomes able to set mysql default charset Horde brokes.

2009-12-15 18:36:19 e (dot) cerny (at) seznam (dot) cz Comment #4 Reply to this comment
there should be something like "escrzya" with diacritics
2009-12-15 18:33:40 e (dot) cerny (at) seznam (dot) cz Comment #3 Reply to this comment
But this way data in database are utf-8 overconverted from latin1 to 
utf8. So instead of "ěščřžýáíéžřč" there is 
"žřčšěščřžýáí"
1. if any other application uses this it is imposible to convert 
"žřčšěščřžýáí" to "ěščřžýáíéžřč" 
because there are ilegal latin1 characters.
2. backup, restoring and moving database is made problematic

I solved this with

         if ($dsn['charset']) {
             if (!@mysql_set_charset($dsn['charset'])) {
                 return $this->mysqlRaiseError();
             }
         }

added to pear/DB/mysql.php connect function after database conect

Which made PEAR function the old way

But this in not a permanent solution. The next "pear upgrade DB" will 
broke it again...
2009-12-15 17:54:50 Jan Schneider Comment #2
State ⇒ Not A Bug
Priority ⇒ 1. Low
Reply to this comment
Doesn't matter because we convert the data to the correct charset anyway.
2009-12-15 17:47:10 e (dot) cerny (at) seznam (dot) cz Comment #1
Type ⇒ Bug
State ⇒ Unconfirmed
Priority ⇒ 3. High
Summary ⇒ $conf['sql']['charset'] is ignored
Queue ⇒ Horde Base
Milestone ⇒
Patch ⇒ No
Reply to this comment
PEAR function DB::connect ignores $dsn['charset'] for mysql.
Database charset defaults to latin1 whatever is set in 
$conf['sql']['charset'].


Saved Queries