bug in character-set handling

[workaholic]

Hi everybody,

I just found a significant bug that magically switches all language-specific characters to ugly nonsense-characters in all blog entries!

But keep reading - I found a fix

Here is my scenario, where I'm able to reproduce this bug easily:

• s9y release: version 1.2
  character-set when installing blog: latin1 (native)

After using s9y a few weeks, I went to

administration -> configuration

to change my blog title.
Right after saving that change, all my umlaut-characters showed up wrong! So I returned to the configuration menu and noticed, that there was UTF-8 configured - which of course did not match my initial setup. "No problemo" I thought and changed that back to 'native'. But no luck! Still no umlauts visible...

After hunting the problem for multiple hours, by restoring my blog from backups and diff'ing the SQL-statements I finally managed to fix the problem!

In the table 'serendipity_config' I changed the value of 'charset' from '' to 'UTF-8/' and voilà - umlauts back to live!

Here's the exact SQL-statement I've been using for the fix:

UPDATE `serendipity_config` SET `value` = 'UTF-8/' WHERE CONVERT( `name` USING utf8 ) = 'charset' AND CONVERT( `value` USING utf8 ) = '' AND `authorid` =0 LIMIT 1 ;

I then tried to reproduce the problem and yes I could - here are my observations:

In the administration->configuration-page...

• Setting the 'Charset selection' option to 'UTF8' writes 'UTF-8/' to the value columen in the above mentioned 'charset' record. It also writes 'utf8' to the variable $serendipity['dbCharset'] which sounds OK.
  Setting the 'Charset selection' option to 'Native' clears the value columen in the above mentioned 'charset' record.<br>
  !!But it does not change the variable $serendipity['dbCharset'] which probably is wrong (in my case 'latin1')!!
  </list>
  
  <b>However, the major problem - at least in my case - was, that the configration-pages comes up with 'UTF8' as the default setting and writes that value even if you change a totally different option!</b>
  It seems like the charset setting of 'latin1' in the MySQL-DB I chose during installation of s9y, is not recognized in the configuration-page.
  And as it doesn't correctly update the charset-settings when choosing 'native' the user ends up in a dead-end!
  
  I found at least one posting of another user (http://board.s9y.org/viewtopic.php?t=10 ... light=utf8) who seemed to have exactly the same problem and got no solution.
  
  hth,
  workaholic.
  
  BTW: it seems as I'm not able to switch the 'Language'-Setting in the administration->configuration page at all...
  Selecting any other language seems not to be saved, as no change is visibile and returning to the config-page keeps showing me 'german'.
  Any clue?

[garvinhicking]

Hi!

Hm, in my config the DB value is changed according to what I entered. However, changing the charset does not properly set $serendipity['dbCharset'], that is true. That value is not stored in the serendipity_config DB table, but in the serendipity_config_local.inc.php file. There the value is drawn from the currently loaded language file, so you sadly need to save your s9y configuration twice so that the right value (of the constant SQL_CHARSET) can be inserted. That's a glitch somehow, I agree, but changing the Charset after installation is definitely a hard thing that should be avoided at all costs, and often requires manual interaction (because transcoding is necessary).

You must change the language in "Personal Configuration", not the blog configuration. There the language only applies to non-logged in visitors.

Regards and thanks for posting here,
Garvin