Welcome! Log In Create A New Profile

Advanced

a challenge-sphider works on all sites exexpt for one on server

Posted by jhodges 
a challenge-sphider works on all sites exexpt for one on server
April 19, 2007 02:16PM
Hello

I have an issue I have spent hours on. I have 12 web sites on a server. All of them sphider fine, except for www.m-media.com. Instead of sphidering the index.php page, it goes to some page I cannot find, with links that no longer exist. I can change the directoryindex to another page, or delete my index page, and the web browser will go to the changes, but the spider always goes to some page I cannot find that has links to /admin and /reports folder that do not exist on the server. (they did once upon a time). I have tried three different php search engines, they all are doing the same thing.

thanks for any suggestions, welcome to try to sphider it.
jhodges@m-media.com
It seems a javascript site. Maybe Sphider has no solution for it or maybe Ando doen't want to show the solution with Sphider
Re: a challenge-sphider works on all sites exexpt for one on server
April 24, 2007 12:31PM
Hello-thanks for your post-Not sure what you mean by "javascript iste". I do have some scripts running (like I do on the other sites that spider, they are all identical except this site has an online store, protected directories, and thawte cert on it) but they are all php pages with top, side, and bottom behing server side includes. thanks for any input
New info
April 24, 2007 07:25PM
Hello

I discovered that the spider is being re-direct to an old IP. If I spider by the new IP, it is still being redirected to the old IP. This is crazy. I checked net solutions, searched all code on site, I cannot find where the redirect is. Any ideas?
Re: a challenge-sphider works on all sites exexpt for one on server
April 25, 2007 06:55PM
Try pinging the domain from the server? maybe the server is using an internal dns or hosts file that is pointing to the old IP?
Re: a challenge-sphider works on all sites exexpt for one on server
April 26, 2007 04:25PM
Found wrong records in DNS. Fixed that. Pings to correct IP. (Wrong records were redirected to old server). Fixing it did not fix the problem-Cleaned out all local cache and flushdns

Is there a cache file for php or sphider that could be storing this old record and sending me back to the old site now that the dns is fixed?
Re: a challenge-sphider works on all sites exexpt for one on server
April 27, 2007 06:28PM
Works now! After they fixed the dns, just took a day to uncache. Thanks for all the suggestions
Sorry, only registered users may post in this forum.

Click here to login