Welcome! Log In Create A New Profile

Advanced

dynamic pages not indexed

Posted by massimopt 
dynamic pages not indexed
April 16, 2007 05:28PM
hi everybody,
I'm a new user and I tried to use sphider, but I have a problem with dynamic pages, that are not indexed in sphider:

18. Retrieving: http://www.istitutopasquini.it/pof.php at 16:25:16.
Size of page: 20.56kb. Starting indexing at 16:25:17. MD5 sum checked. Page content not changed
Links found: 0. New links: 0
this page contains a lot of link to dynamic pages loke this:
<li class="pof"><a href="pof_sezione.php?id_pof=1" alt="Breve premessa">Breve premessa</a></li>

but sphider don't see this.

Any suggestion?
thanks
Massimo
Re: dynamic pages not indexed
April 16, 2007 07:46PM
Sorry, wrong installation, now it's all ok
Massimo
Re: dynamic pages not indexed
April 20, 2007 04:07AM
massimopt Wrote:
-------------------------------------------------------
> Sorry, wrong installation, now it's all ok
> Massimo



Massimo, I'm having the same issue. Can you let me know what you did? Is there a separate install that handles dynamic pages?

Thanks.
Anonymous User
Re: dynamic pages not indexed
April 21, 2007 12:07AM
sphider doen't recognize javascript
Re: dynamic pages not indexed
April 21, 2007 03:33AM
I'm not talking about Javascript.

My page looks like this:
"http://domain. com/index.php?page=home"

But when I try to index it, I get this:
1. Retrieving: "http://domain. com/index.php?page=home" at 20:32:00.
Unreachable: http 404
Links found: 0. New links: 0

Any ideas?

Thanks.



Edited 1 time(s). Last edit at 04/21/2007 03:34AM by peefoo.
Anonymous User
Re: dynamic pages not indexed
April 24, 2007 12:21AM
Maybe you could if the server permitted directory browsing
Re: dynamic pages not indexed
April 24, 2007 12:59AM
Sphyder will not index my dynamic pages either.

When I index, spider.php says "Page contains less than 10 words" for all of my dynamic pages. These pages have well more than 10 words.

What is more, is my dynamic pages don't even appear dynamic as I have used mod_rewrite on them (I had the same problem with Sphyder prior to my installation of mod_rewrite).

All my settings seem to be fine... Hopefully someone can shed some light on this problem...

Thanks in advance.
Re: dynamic pages not indexed
April 25, 2007 03:43PM
OK problem resolved now.

Rescortes was correct in his post above - I needed to enable directory browsing in Apache. Once it was enabled, Sphider found all my dynamic pages.

The change that I made was to add the line "Options +Indexes" (without quotes) to my .htaccess file. If you don't already have an .htaccess file, just make a blank text file in the root folder of your website (or wherever the folder is where you want stuff to be indexed) named .htaccess and put that one line in it.

I hope this helps some of the other people out there with this problem.

Thanks,
phoenixMetal
Sorry, only registered users may post in this forum.

Click here to login