Welcome! Log In Create A New Profile

Advanced

Create sitemap.xml

Posted by Tec 
Tec
Create sitemap.xml
October 14, 2007 06:01PM
This is an example how to create a sitemap.xml. Compatible with http://www.sitemaps.org/schemas/sitemap/0.9 this mod automatically creates sitemaps together with Spiders index, reindex,reindex-all and reindex with erase. According to the site url, as registered in Sphiders admin section, the xml files will be individually named like 'sitemap_www.abc.de.xml' and stored in .../admin/ folder. When running a reindex or reindex-all the existing sitemaps will be overwritten with the actual data set.
If the sitemaps shouldn't be created automatically, (in future versions of this mod) a checkbox in admin section could prevent their production.


In .../admin/spider.php search for:

printStandardReport('completed',$command_line);


Above this row include:


//*** Create sitemap.xml ***
$changefreq = "monthly"; // individualize this variable
$priority = "0.50"; // individualize this variable

// Below this only change something, if you are sure to remain compatible to http://www.sitemaps.org/schemas/sitemap/0.9
$date = date("Y-m-d"winking smiley;
$time = date("h:i:s"winking smiley;
$modtime = "T$time+01:00";
$version = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>" ;
$urlset = "<urlset xmlns=\"http://www.sitemaps.org/schemas/sitemap/0.9\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:schemaLocation=\"http://www.google.com/schemas/sitemap/0.84 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd\">";
$copyright = "<!-- Generated by Sphider with additional module created by Tec (v.1.0 rev.2) -->" ;
$update = "<!-- Last update of this sitemap: $date / $time -->" ;

$all_links = '';
// Get all links of the current domain
$res=mysql_query("select * from ".$mysql_table_prefix."links where site_id = $site_id"winking smiley;
echo mysql_error();
$num_rows = mysql_num_rows($res);

// Create individual rows for XML-file
for ($i=0; $i<$num_rows; $i++) {
$link = mysql_result($res, $i, "url"winking smiley;
$all_links = "$all_links<url><loc>$link</loc><lastmod>$date$modtime</lastmod><changefreq>$changefreq</changefreq><priority>$priority</priority></url>\n";
}

// Create filename and open file
$name = parse_url($url);
$hostname = $name[host];

if ($hostname == 'localhost'){ // if we run a localhost system extract the domain
$pathname = $name[path];
$pos = strpos($pathname,"/",1);
$pathname = substr($pathname,$pos+1);
$pos = strrpos($pathname,"/"winking smiley;
if ($pos) $pathname = substr($pathname,0,$pos);
$filename = "sitemap_localhost_$pathname.xml";
if (!$handle = fopen($filename, "w"winking smiley) {
print "Unable to open $filename";
die;
}

} else { // if we run in the wild
$filename = "sitemap_$hostname.xml";
if (!$handle = fopen($filename, "w"winking smiley) {
print "Unable to open $filename";
die ('');
}
}

// Now write all to XML-file
if (!fwrite($handle, "$version\n$urlset\n$copyright\n$update\n$all_links</urlset>\n"winking smiley) {
print "Unable to write to $filename";
die ('');
}
fclose($handle);

// sitemap.xml done! Now final printout
print "<br>Created: <font color=\"blue\"> $filename</font><br><br>";
print "[Back to <a href=\"admin.php\">admin</a>]<br>";
//*** Created sitemap.xml ***


Happy coding

Tec
Re: Create sitemap.xml
October 17, 2007 12:23AM
Awesome work. This does exactly what I wanted to do. Very useful mod indeed! Thank you
dbi
Re: Create sitemap.xml
October 18, 2007 08:37PM
Is there any way to limit sitemap to 50.000?
Tec
Re: Create sitemap.xml
October 18, 2007 11:35PM
In my mod search for:

$num_rows = mysql_num_rows($res);

Beyond this row additionally include:

$max_rows = '50000'; // include your max. count of rows for sitemap.xml here
if ($num_rows > $max_rows) $num_rows = $max_rows;


Happy coding

Tec
dbi
Re: Create sitemap.xml
October 19, 2007 12:13AM
Oh sorry. I forgot to say. Sitemap limit is 50k, but you can have a lot of files. Can script split urls in 50k maximum links and have as many as necessary files?

Can I generate sitemap without reindex whole site?

Diego
Tec
Re: Create sitemap.xml
October 19, 2007 01:02AM
<<< Can script split urls in 50k maximum links and have as many as necessary files?>>>

At present not forseen. Donations are welcome for individual development.


<<< Can I generate sitemap without reindex whole site? >>>

Yes, would be an interesting option for my mod 'list all pages from a domain'. Version 3 . . .

Tec, tired. Enough for today.



Edited 1 time(s). Last edit at 10/19/2007 01:03AM by Tec.
liz
Re: Create sitemap.xml
September 15, 2008 05:00AM
How do I script Sphider to limit search results to sites modified in the last 24 hour, week, month and year.
Re: Create sitemap.xml
October 05, 2008 07:41AM
hi, just a quick question. Does it make a sitemap per site that is added up to 50,000 or is it up to 50,000 for the total install?
Re: Create sitemap.xml
October 06, 2008 06:51PM
One word: "phenomenal". I have been trying to find something like this for weeks !!!!!

Cheers
Re: Create sitemap.xml
November 10, 2008 04:33AM
Creating a xml sitemap with Sphider-Plus didn't work for me.

I found a great free program though: GSiteCrawler from http://gsitecrawler.com/

which creates the XML locally and uploads it to the server via FTP

1500 MB of disk space - 100 GB of data transfer - all free here: http://www.000webhost.com/91892.html



Edited 1 time(s). Last edit at 11/10/2008 04:34AM by Ranbir.
Re: Create sitemap.xml
November 13, 2008 05:58AM
I use the XML sitemap creator from Link Vendor. You can find it at:

http://www.linkvendor.com

Hummdis Communications
Freelance Website Design/IT & Network Consulting
http://www.hummdis.com
Re: Create sitemap.xml
November 20, 2008 07:15AM
cool thanks for the mod
Re: Create sitemap.xml
February 19, 2009 04:20PM
ok so yeah i added this mod and it goes into my admin section,

but what am i putting into google sitemap here

like am i putting www.mydomain.com/admin then the site maps that are created

or

should there be something i need to edit for it to be comppiled into one site map

some information would be good

thanks
Tec
Re: Create sitemap.xml
February 19, 2009 08:00PM
Hello relisys,

We already discussed this at:
http://www.sphider.eu/forum/read.php?3,4762,4769#msg-4769
Perhaps helpful for you.

Tec
Re: Create sitemap.xml
February 20, 2009 03:24PM
Hi,

I installed this, but noticed this mod just creates sitemaps for the individual sites added to the database, which I dont really need as all my ecommerce sites I have doe this automatically, so how can we create a sitemap which lists all our spidered results on the search engine for seo?

I added the search engine under a seperate domain, as we have over 20 shops based upon heavily modified versions of oscommerce, so we want to use sphider for SEO purposes, *IE more quality inbound links etc..


Cheers
Chris
Sorry, only registered users may post in this forum.

Click here to login