Welcome! Log In Create A New Profile

Advanced

My Sphider Carwler is not crawling / Indexing my websites

Posted by ersachingupta 
My Sphider Carwler is not crawling / Indexing my websites
March 10, 2014 05:54PM
hello
This Sphider Carwler is not indexing /.crawling my websites, kindly help.
When i add a website and try to index it, it gives me following error.
-------------------------------------------------------------------------------------------------
ERROR 403 - FORBIDDEN
Why am I seeing this page?

Is there anything that I can do?

Understanding Filesystem Permissions

How to modify your .htaccess file

How to modify file and directory permissions
-------------------------------------------------------------------------------------------------

My Address bar has this link data " http://crawlnsearch.com/admin/admin.php?f=index&url=http://www.w3schools.com/ "

I am using hostgator hosting.
Tec
Re: My Sphider Carwler is not crawling / Indexing my websites
March 12, 2014 04:48PM
<<< ERROR 403 - FORBIDDEN
Why am I seeing this page? >>>


You need to chmod 777 the admin folder of your Sphider-installation. To be performed by your FTP client.

Tec
Re: My Sphider Carwler is not crawling / Indexing my websites
March 12, 2014 05:50PM
Hi tec
Thanks for reply first of all.
As per your suggestion, I have changed the permissions to 777.
But now i am getting
ERROR 500 - INTERNAL SERVER ERROR
Why am I seeing this page?

Is there anything that I can do?

Understanding Filesystem Permissions

How to modify your .htaccess file

How to modify file and directory permissions

Please help
Tec
Re: My Sphider Carwler is not crawling / Indexing my websites
March 12, 2014 06:39PM
<<< ERROR 500 - INTERNAL SERVER ERROR >>>
Obviously your hoster does not accept full write privileges for the Sphider scripts. You will be obliged to chmod again. Try to set chmod 755
Don't want to frighten you, but running a crawler like Sphider on a 'Shared Hosting' server often is a most problematical situation .
In order to understand better:Your former 403 error message did appear at which situation? While opening the Admin backend, or when starting the index procedure? Please explain in some more details.
Additionally: You are discussing about a .htaccess file. The .htaccess of original Sphider does not contain any critical command. Even hostgator MUST accept these files. For your background info, please use the search form of this forum and query for 'hostgator'. In order to receive all results, you should enable the option 'All dates'

Tec
Sorry, only registered users may post in this forum.

Click here to login