Full-text search doesn't work for multi-byte languages

Found a bug? Tell us!!
Post Reply
deminy
Regular
Posts: 28
Joined: Mon Oct 10, 2005 6:17 am
Contact:

Full-text search doesn't work for multi-byte languages

Post by deminy »

When a user use East Asian words as keywords to search entries, he will get no result since MySQL's full-text search doesn't work for multi-byte languages (like Chinese, Japanese, etc).

I think it's better to provide a switch to switch between full-text search and like-statement search, which means the following two functions need to be updated:

1. function "serendipity_searchEntries" in file "include/functions_entries.inc.php"; and

2. function "serendipity_drawList" in file "include/admin/entries.inc.php"

BTW, there is a minor bug in function "serendipity_drawList". Function "serendipity_searchEntries" works for PostgreSQL, SQLite and MySQL, while function "serendipity_drawList" only works for MySQL (if user wants to search "body" information).
garvinhicking
Core Developer
Posts: 30022
Joined: Tue Sep 16, 2003 9:45 pm
Location: Cologne, Germany
Contact:

Re: Full-text search doesn't work for multi-byte languages

Post by garvinhicking »

Hi!

Hm, but the LIKE search is terribly slow. I figure a better solution would be to make MySQL support multibyte languages. :)

Adding a config option to toggle the use should not be that hard, because I think there already is a code brnach that uses LIKE if pgsql or sqlite is used...

Regards,
Garvin
# Garvin Hicking (s9y Developer)
# Did I help you? Consider making me happy: http://wishes.garv.in/
# or use my PayPal account "paypal {at} supergarv (dot) de"
# My "other" hobby: http://flickr.garv.in/
xzilla
Regular
Posts: 10
Joined: Wed Aug 16, 2006 5:05 pm
Contact:

Re: Full-text search doesn't work for multi-byte languages

Post by xzilla »

Maybe the better solution is to just switch to postgres and use tsearch2, which can do multibyte free text search.
Post Reply