Special characters like accents
Posted by Fernando
|
Special characters like accents April 08, 2007 11:50PM |
I'm spanish and I had problems with the suggest script because it put strange symbols instead of letters like ?????? (aacute, eacute, ntilde...).
It can be easily solved. In /sphider/include/js_suggest/suggest.php find near the end:
And replace it with this:
Now special characteres work well!
www.mundodivx.com
It can be easily solved. In /sphider/include/js_suggest/suggest.php find near the end:
print "new Array(" . implode(", ", $js_array) . "
";
And replace it with this:
print utf8_encode("new Array(" . implode(", ", $js_array) . "";
Now special characteres work well!
www.mundodivx.com
|
Re: Special characters like accents May 05, 2007 01:40AM |
indexing and searching text of languages with accents
Posted by: Tec (IP Logged)
Date: April 26, 2007 12:32AM
No, don't try any other collation. UTF8 includes all worldwide known characters.
I prepared something like homp?ge and nonsen?e as text to be indexed. After indexing, I had no problem to search for these words. Also the Suggest Framework has correct intentions for these words. I used Sphider v.1.3.1.f as original downloaded.
Please disable word stemming. Stem sometimes disturbes correct searching of difficult words.
Would be helpful to know, if the words containing ? or ? are correctly indexed in your database. Have a look at your table $mysql_table_prefix(keywords). They should be stored there.
After changing to utf8_bin, did you index again your sites? Don't use "Reindex", because admin's reindex only looks for changes of the sites to be indexed (MD5 checksum). Reindex doesn't recognize admin- or database-modifications. You need to do a fresh index.
Very hard job if you are indexing a lot of sites. In order to be helpful for this:
Install a new script into folder ./admin/ which is called erase.php with the following content:
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<link rel="stylesheet" href="admin.css" type="text/css" />
</head>
<body>
<?
include "auth.php";
$settings_dir = "../settings";
include "$settings_dir/conf.php";
$erase =array ("domains","keywords","links","link_keyword0","link_keyword1","link_keyword2","link_keyword3","link_keyword4","link_keyword5","link_keyword6","link_keyword7","link_keyword8","link_keyword9","link_keyworda","link_keywordb","link_keywordc","link_keywordd","link_keyworde","link_keywordf","pending";
foreach ($erase as $allthis){
mysql_query ("TRUNCATE `".$mysql_table_prefix."$allthis`";
echo mysql_error();
}
print "<br>Erasing done<br><br>";
?>
<a href='spider.php?all=1'><br>Okay, now reindex all<br><br></a>
<a href='admin.php'><br>Return to admin without reindex</a>
</body>
</html>
Now open /admin/admin.php and search for:
<li><a href='spider.php?all=1'>Reindex all</a><br><br></li>
After this, place a new row with this content:
<li><a href='erase.php'>Reindex all with erase</a><br><br></li>
Ready. In admin section "Sites" now you will find a new item called "Reindex with erase". Use this to do a fresh index without entering all URL's again.
Tec
indexing and searching text of languages with accents
Posted by: Tec (IP Logged)
Date: April 26, 2007 06:50PM
Okay, I'm using JS suggest framework and was discussing about that.
You are right. "Did you mean..." fails with ?cole and nonsen?e.
Modification for this bug:
Open ./include/searchfuncs.php
Search for the row:
result = mysql_query("select keyword from ".$mysql_table_prefix."keywords where soundex(keyword) = soundex('$word')";
and replace it with this:
result = mysql_query("select keyword from ".$mysql_table_prefix."keywords where soundex(keyword) <> soundex('$word')";
Now "Did you mean..." offers ?cole and nonsen?e. For the moment I have not enough time to continue testing. But this is not the final solution, as my system doesn't suggest frivolit? with the above modification....
But this exactly the row witch has to be improved.
Sphider, the never ending story.
Please publish your results here.
Tec
[www.sphider.eu]
Posted by: Tec (IP Logged)
Date: April 26, 2007 12:32AM
No, don't try any other collation. UTF8 includes all worldwide known characters.
I prepared something like homp?ge and nonsen?e as text to be indexed. After indexing, I had no problem to search for these words. Also the Suggest Framework has correct intentions for these words. I used Sphider v.1.3.1.f as original downloaded.
Please disable word stemming. Stem sometimes disturbes correct searching of difficult words.
Would be helpful to know, if the words containing ? or ? are correctly indexed in your database. Have a look at your table $mysql_table_prefix(keywords). They should be stored there.
After changing to utf8_bin, did you index again your sites? Don't use "Reindex", because admin's reindex only looks for changes of the sites to be indexed (MD5 checksum). Reindex doesn't recognize admin- or database-modifications. You need to do a fresh index.
Very hard job if you are indexing a lot of sites. In order to be helpful for this:
Install a new script into folder ./admin/ which is called erase.php with the following content:
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<link rel="stylesheet" href="admin.css" type="text/css" />
</head>
<body>
<?
include "auth.php";
$settings_dir = "../settings";
include "$settings_dir/conf.php";
$erase =array ("domains","keywords","links","link_keyword0","link_keyword1","link_keyword2","link_keyword3","link_keyword4","link_keyword5","link_keyword6","link_keyword7","link_keyword8","link_keyword9","link_keyworda","link_keywordb","link_keywordc","link_keywordd","link_keyworde","link_keywordf","pending"
;
foreach ($erase as $allthis){
mysql_query ("TRUNCATE `".$mysql_table_prefix."$allthis`"
;
echo mysql_error();
}
print "<br>Erasing done<br><br>";
?>
<a href='spider.php?all=1'><br>Okay, now reindex all<br><br></a>
<a href='admin.php'><br>Return to admin without reindex</a>
</body>
</html>
Now open /admin/admin.php and search for:
<li><a href='spider.php?all=1'>Reindex all</a><br><br></li>
After this, place a new row with this content:
<li><a href='erase.php'>Reindex all with erase</a><br><br></li>
Ready. In admin section "Sites" now you will find a new item called "Reindex with erase". Use this to do a fresh index without entering all URL's again.
Tec
indexing and searching text of languages with accents
Posted by: Tec (IP Logged)
Date: April 26, 2007 06:50PM
Okay, I'm using JS suggest framework and was discussing about that.
You are right. "Did you mean..." fails with ?cole and nonsen?e.
Modification for this bug:
Open ./include/searchfuncs.php
Search for the row:
result = mysql_query("select keyword from ".$mysql_table_prefix."keywords where soundex(keyword) = soundex('$word')"
;
and replace it with this:
result = mysql_query("select keyword from ".$mysql_table_prefix."keywords where soundex(keyword) <> soundex('$word')"
;
Now "Did you mean..." offers ?cole and nonsen?e. For the moment I have not enough time to continue testing. But this is not the final solution, as my system doesn't suggest frivolit? with the above modification....
But this exactly the row witch has to be improved.
Sphider, the never ending story.
Please publish your results here.
Tec
[www.sphider.eu]
|
Re: Special characters like accents May 05, 2007 01:42AM |
Special characters like accents
Posted by: Fernando (IP Logged)
Date: April 08, 2007 11:50PM
I'm spanish and I had problems with the suggest script because it put strange symbols instead of letters like ?????? (aacute, eacute, ntilde...).
It can be easily solved. In /sphider/include/js_suggest/suggest.php find near the end:
print "new Array(" . implode(", ", $js_array) . "";
And replace it with this:
print utf8_encode("new Array(" . implode(", ", $js_array) . "";
Now special characteres work well!
Posted by: Fernando (IP Logged)
Date: April 08, 2007 11:50PM
I'm spanish and I had problems with the suggest script because it put strange symbols instead of letters like ?????? (aacute, eacute, ntilde...).
It can be easily solved. In /sphider/include/js_suggest/suggest.php find near the end:
print "new Array(" . implode(", ", $js_array) . "
";
And replace it with this:
print utf8_encode("new Array(" . implode(", ", $js_array) . "
";
Now special characteres work well!
|
Re: Special characters like accents July 09, 2007 06:23PM |
Hi! I'm too Spanish, and Im having lot of problems with this.
I'm using the last version 1.3.2,
print utf8_encode("new Array(" . implode(", ", $js_array) . "";
is done, and partially correct the problem.
to show correctly characters I changed in the header
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
and for "Did you mean:" I changed in search_results.html
<?php echo $sph_messages['DidYouMean'];?>: <a href="<?php print 'search.php?query='.utf8_encode(quote_replace(addmarks($search_results['did_you_mean']))).'&search=1'?>"><?php print utf8_encode($search_results['did_you_mean_b']); ?></a>?
I have deleted all indexs and started again
but I still have problems, specially with the suggest and "Did you mean:"
I have seen you web(Fernando) and the seraches works perfectly, could you help me with it?
I'm using the last version 1.3.2,
print utf8_encode("new Array(" . implode(", ", $js_array) . "
";
is done, and partially correct the problem.
to show correctly characters I changed in the header
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
and for "Did you mean:" I changed in search_results.html
<?php echo $sph_messages['DidYouMean'];?>: <a href="<?php print 'search.php?query='.utf8_encode(quote_replace(addmarks($search_results['did_you_mean']))).'&search=1'?>"><?php print utf8_encode($search_results['did_you_mean_b']); ?></a>?
I have deleted all indexs and started again
but I still have problems, specially with the suggest and "Did you mean:"
I have seen you web(Fernando) and the seraches works perfectly, could you help me with it?
|
Re: Special characters like accents August 16, 2007 10:40PM |
You can adapt this Site search example
(search of "Sphider" in the site "www.sphider.eu"
[www.sphider.eu]
(search of "Sphider" in the site "www.sphider.eu"
[www.sphider.eu]
Sorry, only registered users may post in this forum.