global crawler Any one develop a sphider for global crawling not limited to sites added from admin,please help me http://www.sphider.eu/forum/read.php?3,6733,6733#msg-6733 Thu, 23 May 2013 05:45:45 +0300 Phorum 5.2.10 http://www.sphider.eu/forum/read.php?3,6733,7178#msg-7178 Re: global crawler http://www.sphider.eu/forum/read.php?3,6733,7178#msg-7178 -------------------------------------------------------
> so if i add:
>
> php spider.php -u -l -f
>
>
> Then it will crawl the whole web? Or am I
> misunderstanding the question and the answers?

Yes, until your database implodes...Sphider will NOT index the entire web.]]> Convergence Sphider Mods Sun, 14 Feb 2010 11:59:31 +0200 http://www.sphider.eu/forum/read.php?3,6733,7110#msg-7110 Re: global crawler http://www.sphider.eu/forum/read.php?3,6733,7110#msg-7110 -------------
seo India]]> Anonymous User Sphider Mods Thu, 28 Jan 2010 12:18:40 +0200 http://www.sphider.eu/forum/read.php?3,6733,7001#msg-7001 Re: global crawler http://www.sphider.eu/forum/read.php?3,6733,7001#msg-7001 
php spider.php -u [www.sphider.eu] -l -f

Then it will crawl the whole web? Or am I misunderstanding the question and the answers?]]> alias007 Sphider Mods Fri, 15 Jan 2010 19:21:04 +0200 http://www.sphider.eu/forum/read.php?3,6733,6959#msg-6959 Re: global crawler http://www.sphider.eu/forum/read.php?3,6733,6959#msg-6959 Works great for me.]]> kliopa10 Sphider Mods Thu, 31 Dec 2009 14:42:03 +0200 http://www.sphider.eu/forum/read.php?3,6733,6950#msg-6950 Re: global crawler http://www.sphider.eu/forum/read.php?3,6733,6950#msg-6950 php spider.php -u [www.sphider.eu] -l -f
this then means that the spider will leave the domain]]> jsearched Sphider Mods Thu, 24 Dec 2009 12:28:41 +0200 http://www.sphider.eu/forum/read.php?3,6733,6733#msg-6733 global crawler http://www.sphider.eu/forum/read.php?3,6733,6733#msg-6733 ebadul Sphider Mods Fri, 06 Nov 2009 14:29:13 +0200