Welcome! Log In Create A New Profile


Larger data sets?

Posted by subnet_rx 
Larger data sets?
May 29, 2007 03:49PM
I'm looking to use sphider to actually just get an idea of how many pages are on my company's site. From using other tools, it has to be somewhere around 300,000 pages, but all of them are filling up my RAM to do it, and get bogged down. Why do you say this product is limited to 100,000 pages? Is that just a set limit based on slower performance after that number or a mysql limitation?
Re: Larger data sets?
May 31, 2007 01:52PM
The 100,000 is just a very safe estimate after which things might be slowing up. But it depends heavily on your actual website, your server etc. People have reported Sphider to be working on millions of sites without a problem.
Sorry, only registered users may post in this forum.

Click here to login