Sphider forum - Sphider Support

Re: Sphyder-plus not indexing

jace9 — Mon, 20 May 2013 14:33:46 +0300

Sorry I got confused in your post. You've said that sphider plus is running in your website and same sphider plus running in same website with same wall. I don't think you can see the difference of that?

Re: Sphyder-plus not indexing

snipesearch — Mon, 20 May 2013 12:49:34 +0300

I was not simply critical of ypur script i noted the many ways that the system is superiour to pro and frankly bar the Front end i would preffer to continue using Plus. Its clearly far more feature rich.

Yes sphider plus is runing at [snipesearch.co.uk] and sphider plus is running at [snipesearch.co.uk]

I have been testing them both on the same server.

As it says in the origonal post the issue isnt the script operating front end, for some reason i hit a wall on my shared hostting and it stopped indexing, so i migrated it to a VPS where the issue has continued

Sphider plus successfully reached the following before it stalled indexing

8232 sites + 18695 page links + 0 categories + 433925 keywords + 91376 media links.

Please re-read my post i was not critisism plus for the sake of it i have given an honest comparison. The statement you would probably have a hundred bucks off me already was not supposed to be an insult. I found i got stuck at a number of points when configuring the system and had i simply had the option of paying you five dollars to fix it i would have done that instead.

Please don't take offense to me explaining how things are as i see them.

I shall continue to try to get the sphider plus indexer to resume.

Re: Issues with admin.php and login

sarabicery — Mon, 20 May 2013 11:17:13 +0300

perhaps something in the admin.php file itself

--------------------------------------------------------------------------------

Age of Wushu Gold | Buy RS Gold

Re: Sphyder-plus not indexing

Tec — Sun, 19 May 2013 20:13:58 +0300

Despite of all your critical comments, you are running Sphider-plus v.2.9 on your site . . .

<<< you'd probably have a hundred buck off of me already!# >>>
??? Upps. As to be noticed, it is working quite well at [snipesearch.co.uk] and all the other customer applications.
By the way, Sphider-plus is a 'one man show'.

Tec

Re: Sphyder-plus not indexing

snipesearch — Sun, 19 May 2013 17:40:06 +0300

Hi sorry i came here as i got the spyder pro download from google code as i did not want to pay for renumeration of something before i got it working.

Im having a world of issues on the whole. I have also tried spyder pro.

Sphider pro, is coded in a much neater fashion and indexes faster and easier to style as the way the searcgh is coded is much more versatile and the image search is well developed.

Sphider plus, has a much more sophisticated indexing engine crawls in a much more advanced fashion and ads things to a "PENDING" table when indexing external URLs this is imperitive for a full scale engine.

Sphider plus has a lot more features than either original or Pro and the Foreign language support is fantastic.

Sphider pro is more light weight and seems more compatible across platforms.

I think what it comes down to is if you could have a smaller donation to the project then sphider plus is great tool. Not to mention, if you could pay for bug fixes. The people at sphider plus would make more money, if i could chuck you $5 each time i had a problem , you'd probably have a hundred buck off of me already!#

At this juncture however im not willing to donate to either project for "support" without first knowing that the support is what i need.

Thank you however for your reply.

Snipesearch.co.uk

Re: Sphyder-plus not indexing

Tec — Fri, 17 May 2013 15:21:19 +0300

For all customers, support for Sphider-plus is granted at
[www.sphider-plus.eu]

Tec

Sphyder-plus not indexing

snipesearch — Fri, 17 May 2013 11:40:35 +0300

Hi when i started my SE i was using sphyer then i moved to sphyer plus. I started on shared hosting and am now on a VPS as i needed more resources. After moving the to the VPS, nothing is indexing.

I click index, start the index and then get

Indexing sites . . .

Start indexing at 12:33:22.

and nothing happens, does anyone know why this might be or have a way of tracing the problem? I know sphder plus is a splinter project but its roots live here so im king of hoping someone might be able to help

the website is snipesearch.co.uk

Re: Confused how to exclude all URLS with a specific GET variable

CarolWP — Tue, 14 May 2013 20:16:52 +0300

If you want to exclude those URLs, as opposed to just stripping the session ID, it's actually simpler than you are making it. In the Sites tab, click on Options, then Edit. In the "URLs must not include:" box, put "?osCsid=" (without the quotation marks).

Re: Making Indexing Faster

Tec — Fri, 10 May 2013 10:41:13 +0300

No, multiple db's are available only for Sphider-plus.
By the way, also indexing speed is increased, as only a part of the complete content needs to be erased & re-indexed.
Also, during indexation the search engine remains usable for the search user, as most db content is still searchable. More details about multiple database support are available here:
[www.sphider-plus.eu]

Tec

Re: Making Indexing Faster

solarsurfer — Fri, 10 May 2013 08:13:19 +0300

Hi
Is there a mod to update V1.3.x to accept multiple db ?

Re: Making Indexing Faster

Tec — Thu, 09 May 2013 23:07:29 +0300

<<< But each installation, only had access to certain sites ? >>>
That's exactly the problem.

Assuming indexer 1 is creating a keyword link relationship like (example)
keyword: 'sphider' found at link [www.sphider.eu] == link no.1

and indexer 2 is creating another keyword link relationship like
keyword: 'sphider' found at link [www.sphider-plus.eu] == link no.1

1. Problem: how to merge the different indexation into one database? Because the MySQL server does not accept such duplicates in one table.

2. Problem:
Afterwards the search algorithm will be unable to allocate the search result for a query like 'sphider' to a well defined link.

Of course there might be some new ideas. But for the last 6 years I did not find a suitable solution. Thus I decided to go the contrary way. In order to reduce server load and minimize the response time for query inputs, I am using multiple databases. Most important advantage: the content of each involved database is much smaller. And this is recognizing the user of the search engine, as results are available much faster.

Tec

Re: Making Indexing Faster

solarsurfer — Thu, 09 May 2013 22:02:43 +0300

What if several installations only worked on re-indexing ?
But each installation, only had access to certain sites ?

Re: Making Indexing Faster

Tec — Thu, 09 May 2013 21:15:34 +0300

<<< is there a safe way to use various crawlers on one database ? >>>
Will not be possible to index with several Sphider installations into one database. Because the keyword-link relationship must be unique. Each crawler would try to define its own definition. Consequently the common database would contain duplicates, referring on different links.

Tec

Making Indexing Faster

solarsurfer — Thu, 09 May 2013 20:57:53 +0300

Hi
Anyone know how to use multiple installations of Sphider (Original Version) to increase the speed of Indexing ?

I would assume that it would be easier to make several installations, that then update already indexed data, rather than to go out and crawl new data?
Also can this be done from same machines, as I gather most of the power is used by processing speed, rather than internet speed !

I would assume that you would have to update a master database, verify the data, then replace the old index ? Or is there a safe way to use various crawlers on one database ?

I ask this question, because I need around 20 sites (out of around 4000) to be updated every 3 hours, and 1 crawler doing an update won't do !

Currently I'm running xampp on a windows laptop, while indexing before uploading of the db, to a shared server. I am currently also playing with a Linux server in my front room, in an effort to understand what I want from a dedicated server. My host, like many, only gives me around 30 secs of processor time, at a go, so that won't work !

Any ideas, and anyone that has successfully run multiple spiders / crawlers I would welcome how you have done it, and how successful it is, together with problems you came across !

Thanks guys in anticipation of your reply !!!

Re: How can I change the language of "did you mean"? (SOLVED)

Darvather — Thu, 09 May 2013 05:56:55 +0300

Hi TEC,

Thank you so mucho for help, IT WORK!!!

For other users: If you're trying to change the language of the phrase 'Did you mean' for other like 'Quisiste decir' here is how to do it:

Step 1: Open your administrator and in 'General settings' put this values:
Spanish Language (applies to search page)
dark Search template

Step 2: Go to: …/sphider/languages/es-language.php
never change any of values on the left part (of the array)
Your last row needs to be set to
"DidYouMean" => "Quisiste decir"

Step 3: Go to: …/sphider/templates/dark/search_results.html and:

Around line 8, delete:

Did you mean: ?

and replace it with:

: ?

This will make that in your URL 'Did You Mean' gets replaced with the phrase 'Quisiste decir'

NOTE: THIS WILL ONLY WORK WHEN USING THE DARK TEMPLATE, I don't know about standard template since I don't use it.

Hope it helps any one out there.

I get to this solution thanks to TEC, without his valuable support, patience and help this wouldn't be possible. THANK YOU TEC.

Re: Apostrophes in Sphider Suggest = not found

Tec — Tue, 07 May 2013 17:12:50 +0300

The suggest framework is using another search algorithm, than the main algorithm, which is responsible to create the search results. Consequently they are producing independent results. Additionally the main search algorithm of original Sphider does not accept any special characters like dots, hyphens, apostrophes, etc.

Tec

Apostrophes in Sphider Suggest = not found

BrentEades — Tue, 07 May 2013 17:01:47 +0300

Here's an odd one.

Whenever Sphider Suggest shows a keyword that contains an apostrophe, clicking on that link always produces a 'not found' error. For example, if the suggestions are:

Toronto 4 results
Toronto's 1 result

I'll get:

'The search "toronto's" did not match any documents'

Suggestions?

Re: Issues with admin.php and login

david916ennis — Tue, 07 May 2013 09:34:33 +0300

Awesome! Thanks so much n-jin.tk for posting this helpful tip. (tu)

Medical Services Billing

Re: Still have "mysql has gone away" Error!

Tec — Mon, 06 May 2013 10:11:22 +0300

Using the original Sphider with huge databases, installed on 'Shared Hosting' server, might become a never-ending story.
Please think about using Sphider-plus. In front of the original Sphider, this software offers the following benefits:
- Dozens of ping command are implemented to keep alive the connection to the MySQL server.
- Individual 'Erase & Re-index' for each URL entered in the 'Sites' menu of the Admin backend. Thus not the complete database will be re-indexed, but only a part of the db is renewed
- Admin definable option to interrupt the index procedure after indexing x links. So, you may control the max. runtime for the script, indexing a limited count of links. The index procedure could be manually continued, until all links of a site are indexed.

Tec

Re: Why was my post deleted

dogturf — Mon, 06 May 2013 06:04:10 +0300

Maybe, your post brokes the rules of this forum.

__________________________
synthetic turf

Re: Still have "mysql has gone away" Error!

libertux — Mon, 06 May 2013 02:31:57 +0300

hi,
yes i'm running it on a shared hosting server, so is there some thin i should do?
THANKS

Re: Still have "mysql has gone away" Error!

Tec — Sun, 05 May 2013 23:15:13 +0300

Question:
Are you running Sphider on a 'Shared Hosting' server?

Tec

Re: How can I change the language of "did you mean"?

Tec — Sun, 05 May 2013 10:05:10 +0300

<<< I'm understanting that the reason why I can't change 'Did you mean' for 'Quisiste decir' is that I'm missing this row, am I right? >>>

Yes

<<< If so, could you please tell me in which row I have to place this missing code?>>>

In your search_results.html

Delete

Did you mean: href="">?

and replace it with

: ?

Tec

Still have "mysql has gone away" Error!

libertux — Sun, 05 May 2013 05:49:57 +0300

Hi, and thanks for this awsome script,
I successfully integrate Sphider with CodeIgniter you can see it in action here (http://www.farkess.com), and it's work as a charm, but thease days i have "mysql has gone away" when trying to Reindex Sites.
I folow all the steps in :

[www.sphider.eu]

But still have the same error!

Can some one help me fix it?

Thanks

Re: How can I change the language of "did you mean"?

Darvather — Sun, 05 May 2013 01:32:39 +0300

Hello TEC,

First, I really appreciate all your help and patience with my issue. In regards my last post, I didn't knew I met a bug. All I did was download the script from this site and followed the instructions to installed. Since my web is dark that's why I'm using the 'dark' template. So, sorry if I caused you any confusion. If you want to take a look at it here's my URL: www.ntmexico.com

So, if …/templates/dark/search_results.html is supposed to contain the row
Did you mean: href="">?

and as you saw in the code I posted for you in May 02 is not there, I'm understanting that the reason why I can't change 'Did you mean' for 'Quisiste decir' is that I'm missing this row, am I right?

If so, could you please tell me in which row I have to place this missing code?

Thanks again for your time and help.
Darvather

Why was my post deleted

n-jin.tk — Sat, 04 May 2013 11:16:13 +0300

Re: Issues with admin.php and login

n-jin.tk — Sat, 04 May 2013 08:55:39 +0300

I would suggest for anyone having this problem to upgrade to the newest version of php. if you still can't get in, try logging into your database and looking for your username and password in there. i haven't had this problem myself, and i think its mainly due to the fact that i have the latest version of php installed. it may be different since i am using multiple instances of sphider to crawl the entire web.

Sphider is indexing the entire site even though I set it to only index a sub directory

PHP_User — Fri, 03 May 2013 21:06:16 +0300

I am testing sphider on our testing server. My root is 192.168.2.2/site but I dont want to test it on the entire site (because its faster to test it out on a sub-directory, so I set it to 192.168.2.2/site/events. And I have that path for the index as well. But when I do an index, it starts at the root of the site? I even truncated all the database tables, and started over (to ensure that I was starting the index from scratch) and it still starts indexing at the root. Or it will begin and index and stop after a second and tell me:
" MD5 sum checked. Page content not changed
Links found: 0. New links: 0"

Any ideas?

Re: How To Remove: Table 'sph_search.query_log' doesn't exist

Rubel — Fri, 03 May 2013 18:18:21 +0300

Thanks for your Solution. I already solved this problem.

Re: How can I change the language of "did you mean"?

Tec — Fri, 03 May 2013 09:51:32 +0300

Your above posting is not quite correct, because you met a bug in original Sphider.

As per default download
…/templates/dark/search_results.html
contains the row
Did you mean: href="">?

Instead …/templates/standard/search_results.html
contains the row
: ?

So, you were always referring on the 'dark' template, while I was working with the file in 'standard' template

Tec