Welcome! Log In Create A New Profile

Advanced

Sphidering .htaccess protected sites

Posted by jossif 
Sphidering .htaccess protected sites
November 04, 2008 02:06AM
Sphider uses the fsockopen() function to get files that are spidered. This means that if the site you are spidering is protected via .htaccess or the Apache directive to protect realms, Sphider will return a “401 unreachable” error when attempting to fetch files during spidering and indexing

To enable Sphider to access files in a protected realm, we need to modify the functions url_status and getFileContents.

See the mod here:
http://emandtee.com/blog/2008/11/sphider-and-htaccess-protection/

-- Jossi
Sorry, only registered users may post in this forum.

Click here to login