Welcome! Log In Create A New Profile


has any one got a multithreaded version, that can index multiple domains at same time

Posted by richard_bailey 

I am looking for a multithreaded version, if any one has got any, which can index multiple domains at same time.

currently the spider only allows one url of top domain to be passed, has some one got the code to pass multiple domains at the same time to index.


I posted an answer/possible solution to this question in another post.
@PiOnEeRReSeArCH do you mind giving a link to that post please so that we can see it?
In case anyone is still watching this thread.
I have some what of an edit coming soon that does multiple indexes.
Read more here.
I am going to test this out. I am very tired of the timeouts that happens when indexing large sites. Soon I will be setting up on my own severs. This is insightful option. I will let you know if I am able to get it integrated.

Can you think of special settings other than the permission settings?
Sorry, only registered users may post in this forum.

Click here to login