Welcome! Log In Create A New Profile

Advanced

Sphider Stops Indexing Randomly

Posted by DarinAvatar 
Sphider Stops Indexing Randomly
August 23, 2011 09:58PM
I am creating an index of a website for a work project, and we are currently trying to index the website that the search is for. So far, we have been able to index about 10,000 keywords and a couple hundred links, but sometimes while indexing, it will randomly stop and won't update any further, but I know there's more to index, because the next time I initiate indexing (not re-indexing), it adds more and I occasionally see a "duplicate page" message.

But now, we can't even index. At the top of the indexing page, it says "Connection Refused" and it won't continue. Is there some reason why it won't continue? The database is fine and neither the website we're indexing, nor the website the indexer and search is hosted on, is down, so I don't understand what the problem is. The website is very large, though, and I'm wondering if that's the problem.

Any help is appreciated.
Re: Sphider Stops Indexing Randomly
September 25, 2011 09:53AM
Sphider have problem with indexing large pages, if page more than 1-2 Mb u can just run out memory for php and it stop indexing this page.

Try to restart u webserver (apache?). Also try to run sphider.php from console like php sphider.php -all

[code.google.com] - free for all | support [www.sphider.eu]
Sorry, only registered users may post in this forum.

Click here to login