Sphider forum - Sphider Mods

http://newerahatsdeal005.webs.com/ cheap fitted baseball hats (no replies)

cletaoxitte — Thu, 20 Jun 2013 12:31:13 +0300

[newerahatsdeal001.webs.com] custom baseball hats new era 1 small associated most loved greens dressing up the pricetag
1 flask out of popular greens outfitting are priced
Even the present day motion picture time wasted with regards to 3D works hard now. Film production company has a lurid comic take a check furthermore principal points apply all over this country and even slo-Mo junkies show everything held in your psychedelic perfect form. Ceiling freakouts this gardening shall be joyful since.
baseball hats for men cheap fitted baseball hats 3 steps to making teas
3 the steps to making coffee
Abusive crime reduction are suitable to be your first concern if you use laptop, And moreover simply most of the content is just wise practice. Add boast, Along with promote there is a laptop pc you own, Try to transport your current laptop you are buying to neo-Average laptop or tablet computer baggage which render it observable could possibly have one. Privided you can match up it inside your attache case, Or perhaps a arm gear you're much happier from becoming aim for,
youth baseball hats usa baseball hats 3 your daily life provide training for at change trail location
3 your own life coach at passage train stop
Retailers. appear unable and consequently weighed down. these days. Additionally, This information is going to offer you some advice suitable for accidental injury lawyer policies, And it's a must to keep readin. A number spaces, Toilets several house-hold rooms and dog dens often times have no roof light weight in any way, But apply factors such as floor lights furthermore lamps.

No description in results (no replies)

djsmc — Tue, 23 Apr 2013 02:50:12 +0300

Hi, I've just downloaded Sphider and been testing it on my site. Installation went fine and site was indexed without any issues. However when I do search the results return but there is no description or hilighted search term. Did I install it incorrectly?

Thanks,
Craig

spider video and images (no replies)

coolcurrent — Fri, 19 Apr 2013 23:20:14 +0300

hello,

i have an intranet website that has pages with images and video files
is there any mod for spidering images and video files?

thanks

ad mod (1 reply)

coolcurrent — Thu, 11 Apr 2013 07:51:03 +0300

is there any ad mod? i wat to diaplay test and or image add in the right hand side of the search page

how i do going my site to search engen (no replies)

mayupress — Fri, 08 Mar 2013 18:36:13 +0200

can you please let me know what i do now i just created a id.

Making search case-insensitive (no replies)

halfacre — Thu, 07 Feb 2013 18:05:58 +0200

My searches are currently case sensitive, and I do not want them to be. Is there a simple way to "turn on" case sensitivity in the admin panel? I have not been able to find a way to do this. My knowledge of PHP and mySQL is minimal, just enough to get Sphider running on my site. But this case sensitivity thing is a deal breaker with the number of all-caps acronyms our visitors search for, usually entered in lowercase. Any advice or direction would be greatly appreciated.

More Mods for Sphider (no replies)

memyidesu — Wed, 06 Feb 2013 00:18:47 +0200

Yo Peeps,

I have some more mods that are in the works for the Sphider.

These are some of them:

IP Banning
Domain Banning
New Captcha for the ADD URL page
Pending Index List
IP and Domain Ban check for URL Suggestions

All of these and a few more. And guess what! It will be FREE and on the newest version 1.3.5. I also plan on putting the entire thing in a package for download.

Just because this is FUN!!!

P.S. I am also going to take a bunch of mods from this forum and put those in too. For FREE!!

sphiderpro - free download and review (no replies)

Tec — Fri, 11 Jan 2013 18:34:49 +0200

As Sphider Pro is available as free download for a while now at
[code.google.com]
and also at
[sourceforge.net]
I felt free to do so.

Here is my review on the product.

Summarization: Very disappointing.

Statement of reasons:
Announced to take over from Sphider-plus with big Tam-Tam and a lot of polemic, the Sphider Pro scripts are still very close to the original Sphider, which was released in 2009. As I already stated in another posting:
: . . . it seems they just added some old mods like 'Follow sitemap.xml' and 'Erase & Re-index' to the original Sphider. I've published all these mods in the original Sphider forum several years ago. Adding them to the original Sphider, now they call it Sphider Pro. Okay, in order to remain fair: they also implemented some ideas from Sphider-plus. . . .

But in front of the original Sphider the most important improvements are missing:
- No UTF 8 support. Thus, unable to search for queries in Cyrillic, Japanese, German and other languages.
- Indexing of iframes is not supported.
- Not indexing any kind of feeds.
- No support for IDN URLs.
- Uable to search with wildcards.
- Suggesting to index PDFs, DOCs, etc, the regarding converter are not included.
- Suggesting to contain several search modes, nothing of it is implemented.
- Suggesting to index images, not working.
- Multiple database support is not jet implemented, just separated permission for admin and user in one db.
- Secondary sitemaps are not supported. As result, no links are found.
Application hint: Useless to uncheck the regarding option in Admin settings. The scripts always follow the links in sitemap.xml. It is not that I want to debug the scripts, but please boys, it is not enough to set and unset the variable $follow_sitemap in your config file. You additionally need to use this variable in your script …/admin/spider.php

By the way, also the well-known bugs of original Sphider, for example in category search, are still present.

Regarding the documentation: Thank you for leaving 'Sphider-plus' as part of your poor documentation. Thus, all users are well informed from whom you copied it. Because of my limited English knowledge, even my grammar errors remained!

To all early adaptors: The release dateof Sphider Pro version 3.1 is defined in the docs to Nov. 26, 2012. But having a look at the 'last modified' date of the scripts: Often their date and time is newer than November 26. Are they still debugging their first official release? In order to get always the latest debug version, it might become necessary to write a small batch job for a daily download. Perhaps in the next future the bugs in 'follow sitemap.xml' will become fixed?

It is a great pity that I wasted my time with these scripts. I really was keen to learn from the scripts, which pretend to take over from Sphider-plus. But as in the past, I will be obliged to do it all by my selves.

Tec

Can't save in settings (1 reply)

sugar_ray — Sun, 23 Dec 2012 12:18:03 +0200

Hi, I'm a newbie, so please be gentle!

I have setup SPHider for the 1st time. Everything has gone well but I have a small problem. In the Admin Panel/Settings it wont allow me to save new settings i.e. Administrator e-mail address ???

Sphider-plus version 2.9 released (8 replies)

Tec — Wed, 02 Jan 2013 00:42:15 +0200

As it might be of interest also for the users of original Sphider:
Meanwhile the latest development of Sphider-plus is available at [www.sphider-plus.eu]

In front of version 2.8 of Sphider-plus the following items have been added / modified:

New feature:
Support for non-ASCII URLs using 'Internationalized Domain Names' (IDN).
It is a standard described in RFC 3490, RFC 3491 and RFC 3492.
If activated, internationalized domain names like 'http://президент.рф/' and 'http://müller.de/'
will be accepted as new sites in Admin backend, as well as in User's addurl form.

New feature:
Support Punycode URLs like [xn--90aoqlh7c4a.xn--d1abbgf6aiiy.xn--p1ai]
Converted into the readable form http://события.президент.рф/
To be activated in Admin settings.

New feature:
Besides the usual HTML elements , also delete from full text all those HTML elements,
which are defined like < element >
To be activated in Admin settings.

New feature:
Index only parts of a page, defined by . . .
This feature is foreseen to cooperate with the new HTML5 elements like
section, nav, aside, hgroup, article, header, footer, etc
If enabled in Admin settings, the values as defined in the list-file
…/include/common/elements_use.txt will be used to index only the page content between
. . .

New feature:
Ignore parts of a page, defined by . . .
This feature is foreseen to cooperate with the new HTML5 elements like
section, nav, aside, hgroup, article, header, footer, etc
If enabled in Admin settings, the values as defined in the list-file
…/include/common/elements_not.txt will be used to remove the content between
. . . from the page content.
This is the contrary function to 'Index only parts of a page, defined by . . . '

New feature:
Index only files and documents with defined suffix :
If activated, all pages of the site will be searched for links,
but only files with suffixes as defined in the docs list will be indexed.

New feature:
1. Perform a WHOIS check for sites waiting for approval in Admin backend.
2. Perform a WHOIS check for suggested URLs direct in the addurl form,
so that invalid URLs will automatically be rejected.
For both tests a basic list of WHOIS servers for the generic top level domains
and some important country codes (supporting 30 suffixes),
or an extended list (supporting 155 suffixes) are selectable.

New option to be activated in Admin backend:
Crawler can leave domain during index procedure, but only for canonical links.
Only the canonical link will be indexed, but links found there will be ignored.

New feature:
Obey the 'refresh' meta tags as part of HTML headers.
Now following the redirection and delayed indexing.

New option:
Support UTF-16 coded sites. Will convert UTF-16 coded sites into UTF-8.
To be activated in Admin settings

New option:
For index procedure always use the standard Firefox HTTP_USER_AGENT string
and ignore the individual defined Sphider-plus string. To be activated in Admin backend.

New feature
Follow redirections, which are invoked by JavaScript, when sent as HTTP content.
Will obey directives like:

New feature:
Follow URL redirections caused by HTTP 301, 302, 303 and 307 status codes.

New feature:
Separated PDF converter supplied for 32 and 64 bit Operating Systems.

New feature:
Follow links placed in JavaScript files. Will detect and follow links like
document.write(' All news 2012 ');
Also the complete content of
document.write( this text in all rows');
will be indexed and stored as keywords in db.

New feature:
Now indexing also sites, which do send a obligatory request for a cookie, to be set by the crawler.

New feature:
In order to reduce transmission time, the crawler now requests gzip-formatted data transfer
from the remote server for the URL to be indexed.

New option:
In order to convert the text into UTF-8, use the charset definition as supplied via HTTP by the client
server.
If this option is not activated in Admin Settings, the charset will be extracted from the header of the files to be indexed. If not found, like in PDF documents, the preferred charset will be used.

New option:
Delete duplicate parts of the URL path found in the indexed page URL and the new links.
Unfortunately some CMS seem to be unable to build up a correct path for relative links.
If activated in Admin backend, these duplicate parts of the path will be deleted from the link URL. Should be activated only, if sites are indexed created by dedicated CMS.

New feature:
Show summary of actually active User database at the bottom of result listing.
To be activated in Admin backend, the count of sites, categories, page links and keywords
are displayed.

New feature:
Automatically deleting invalid URLs from Admin 'Sites' view.

Improved 'Add site' function in Admin backend.
Now treating URLs with and without 'www' as equal, and excluding them as duplicate sites.

Improved image indexing procedure
Now also indexing phpBB images, linked by php command files.

New option
Suppress the file suffix from image file names for indexing.

Improved media indexing procedure
In case of missing title tag, now the alt tag is used to define the name of the media.
In case that also the alt tag is missing, the file name will be used as keyword.

Improved "banned domain" management
Now holding name and suffix of the banned domains, and no longer the URLs.

Improved index procedure
Now ignoring links that try to link to the calling URI (self back linking).

Improved link detection for relative links, which are to be found in full text.

Improved input protection against SQL injections

Improved Admin statistics
Now providing also the IP, country code and country name for
- Search log
- Most popular searches
- Most popular page links
- Most popular media links

Updated GeoIP database, used to provide the IP, CC and country name for the Admin statistics.
Now also supporting IPv6 URLs.

Support on Windows systems temporary removed for ppt files, as the converter causes failures on large PowerPoint documents.

Bug fixed, which prevented category selection without activating the "Advanced search form" option.

Bug fixed that caused invalid URL encoding in result listing.

Bug fixed causing the error output "Unknown column 'naame' in field list" during media indexing.

Bug fixed that caused MySQL warning messages during index procedure at some older MySQL versions, if the URL to be indexed contained blank characters.

Bug fixed, which caused invalid URL creation for relative links containing a file name and/or query.

Bug fixed in option 'Crawler can leave domain'.

Bug fixed in option 'Use list of div ids to ignore the div content during index/re-index'.

Bug fixed in option 'Enable to decode entity coded sites into standard HTML characters'.

Bug fixed in 'addurl' form, which prevented input of words containing accents in 'title' and 'description' fields.

Some additional small bugs killed.

[Help] Index RSS (1 reply)

nort0n — Tue, 26 Mar 2013 17:13:21 +0200

I would like to use and RSS to indexes last page, how can I do that?

Thx you

need help to move my database (1 reply)

moin5050 — Tue, 23 Apr 2013 02:05:40 +0300

hey i am learning to program php and dont know any thing about sql ,just little knowledge of both.
i like sphider very much because it is open source and it does index for site that i wanted.

so i installed sphider on my pc and index approx.....10 sites..
now i want to copy that database and upload on my site but i am having a problem that it database doesnt import any tables....
i think i am making mistakes in basic.

so please could you help me....the step i should follow.

thanks you in advance....

URL Filter for Add a url page (1 reply)

memyidesu — Tue, 04 Sep 2012 00:03:08 +0300

Yo Peeps,

I finally figured out a decent way to filter some of the garbage from the Add a URL page. This should be added into the code of the 'addurl.php' page just below this line:

if ($compurl['path']=='') {
                $url = $_POST['url']."/";
            }

Then you put this in:

//Parse the URL for fakes
			if ($url == "/"){
				die ("A URL contains more than ONE character.");
			} else if (substr($url,0,7) != "["];){
				die ("READ THE INSTRUCTIONS!!! You need to include the prefix 'http://'.");
			} else if (substr($url,0,8) == "[];){
				die ("Nice try. That is not a URL.");
			} else if (substr($url,-6) == "[/url]") {
				die ("This is not a forum, DIP****!!!");
			} else if (substr($url,-3,1) != ".") {
				if (substr($url,-4,1) != ".") {
					if (substr($url,-5,1) != ".") {
						if (substr($url,-6,1) !=".") {
							die ("READ THE INSTRUCTIONS!!! TOP LEVEL DOMAINS ONLY!!!");
						}
					}
				}
			}

Feel free to change the text to your liking.
P.S. Remeber that where you see the smiles you need SHIFT + 0
P.P.S. Edited because I founf a little better way to do it.

fr-language.php (no replies)

grandebou — Sat, 25 Aug 2012 23:32:22 +0300

Hi all,

Here is a French language file with correct accents and spelling.

 "Catégories",
        "CATEGORIES" => "CATEGORIES",
        "Untitled" => "Document sans titre",
        "Powered by" => "Grâce à",
        "Previous" => "Précédent",
        "Next" => "Suivant",
        "Result page" => "Page de résultat",
        "Only in category" => "Seulement dans la catégorie",
        "Search" => "Chercher",
        "All sites" => "Tous les sites",
        "Web pages" => "Pages Web",
        "noMatch" => "La recherche \"%query\" ne correspond à aucun document",
        "ignoredWords" => "Mots vides ignorés (trop court ou commun): %ignored_words",
        "resultsFor" => "Résultats pour:",
        "Results" => "Présentation des résultats %from - %to parmi %all %matchword (%secs secondes)", //
        "match" => "résultat",     //
        "matches" => "résultats", //
        "andSearch" => "Recherche ET",
        "orSearch" => "Recherche OU",
        "phraseSearch" => "Recherche d'expression",
        "show" => "Montrer ",
        "resultsPerPage" => "résultats par page",
        "DidYouMean" => "Voulez-vous dire"

);
?>

Me Search Engine Project (no replies)

Hablow — Thu, 26 Jul 2012 02:36:03 +0300

Hello friends, my search engine project is opened.

Demo : [hablow.com]

Im waiting your comments :)

Removing links from search page (no replies)

Sphider_Tamer — Sun, 22 Jul 2012 15:34:54 +0300

Hi Guys and Gals,

I've been pulling my hair out with this one and wondered whether anyone might be able to help.

I need for my users to be able to remove irrelevant links from the search page. So next to each search result I want a link that is a little something like

Quote
Remove Page

My issue is that I don't seem to be able to define the link_id for each record that is pulled. I have scoured and tweaked my way through searchfuns, search, commonfuncs and search results, but to no avail. Anybody have any ideas?

This is a really simple problem to be holding me back and I'm sure I'll kick myself when someone points it out.

Thanks in advance.

URL Banning feature (3 replies)

memyidesu — Fri, 25 Jan 2013 23:34:22 +0200

Yo Peeps,

I've been hard at work on this one. Who would like a URL banning feature to go with the ADD URL feature that I modified?

Update: Almost finished. I'm running the final tests now.

Sphider functionality and adding a sitemap (1 reply)

SgtMash — Thu, 19 Jul 2012 01:06:53 +0300

Hi there,

Has anyone used / got any suggestions for whether or not you can use Sphider to auto-generate a sitemap for you?

I've been asked to integrate an automatic sitemap for a client who is using Sphider for the search functionality on their site and think it would be an elegent solution / could share a lot of functionality already included in Sphider?

If this isn't possible, could anyone suggest a good script / solution to use along side Sphider to creatw / maintain a sitemap?

Thanks for any help/ suggestions.

Nick

CATDOC for Windows (1 reply)

santosh — Mon, 25 Mar 2013 20:38:37 +0200

Hi,

There is a link mentioned in the documentation for downloading catdoc.exe for windows, but the links is not working. Can any please share if you have any working link or mail me the exe file if you have.

Thanks
Santosh

Sphider not indexing local files (1 reply)

dtaylor7@btconnect.com — Tue, 26 Jun 2012 09:28:00 +0300

I have tried everything I can think of to do this.

I've Sphidered: [127.0.0.1] , I've tried HTTP:/192.168.0.X/site_dir , I have a static IP so I tried that too but to no avail. My Sphider set up is pretty much bog standard and I can Sphider files on remote machines without a problem. I'm running UBUNTU and the Apache server. Anyone got any other suggestions?

Beetle Search - Version 1.0.0b (3 replies)

Sierowski — Sat, 21 Jul 2012 01:55:24 +0300

This is the first version of beetle search version 1.0.0 based on sphider version 1.3.5
Beetle Search 1.0.0b

take a look :-)

Adding checkbox to url list (no replies)

cladiron — Thu, 31 May 2012 01:17:43 +0300

Please Delete this post.
I reposted in the correct forum.
Sry for the bad post location.

http://www.sphider.eu/forum/read.php?2,9471

Does anyone know how to add checkboxes next to each domain url in the admin.php ?

This would allow an easier option to delete some urls without having to go into each 1.
It would be nice if there would be the option to delete the info the urls index or leave it, but not required.

I would like to add this feature to the Theme version i have here.
http://www.sphider.eu/forum/read.php?3,9239

Thanks guys.

Keywords mode ? (no replies)

euandrei — Mon, 14 May 2012 02:35:07 +0300

Can anyone make a simple mod in which :
Admin turn of robots, and he adds kewords manually and description manually ?
I bought sphider plus, but i am not satisfied. To complicated
Thank you, multumesc

[MOD] Maxvalue v1.0.0 (no replies)

Sierowski — Sun, 13 May 2012 21:33:55 +0300

Sphider v1.3.5 Modification
---------------------------

Name And Version
----------------

[MOD] Maxvalue v1.0.0

Description
-----------

This mod will stop the user inputting to many characters in the search box.

01. add the following attribute to the searchbox input element maxvalue="256"
this effects the client side only.

eg.

By David Sierowski

[MOD] Set Focus v1.0.0 (no replies)

Sierowski — Sun, 13 May 2012 20:46:49 +0300

Sphider v1.3.5 Modification
---------------------------

Name And Version
----------------

[MOD] Set Focus v1.0.0

Description
-----------

This mod will give focus to the search input box and allow
the user to start typing without clicking on the search box.
the code should be placed in the footer.html file in your
templates folder before the closing body tag.

CODE.TXT

By David Sierowski

[MOD] Reset Button v1.0.0 (no replies)

Sierowski — Sun, 13 May 2012 20:44:21 +0300

Sphider v1.3.5 Modification
---------------------------

Name And Version
----------------

[MOD] Reset Button v1.0.0

Description
-----------

This mod will add a reset button to the template giving the
user the ability to reset the search box. Enter the code from code1.txt after
the submit button code. simple but effective.

CODE.TXT

By David Sierowski

[MOD] Redirect v1.0.0 (no replies)

Sierowski — Sun, 13 May 2012 20:42:09 +0300

Sphider v1.3.5 Modification
---------------------------

Name And Version
----------------

[MOD] Redirect v1.0.0

Description And Installation
----------------------------

This mod will redirect to the search.php page if there is no index.html,
index.htm or index.php file in the root directory of your sphider
installation.

Just copy the files index.htm, index.html and index.php to the root folder
of your sphider installation.

INDEX.HTM, INDEX.HTML, INDEX.PHP - Note these files are all the same.

Redirecting...www.w3.org] type="text/javascript" language="JavaScript">function r(){window.location="search.php";}setTimeout('r()', 3000)Redirecting...
Redirecting to the search page. If the search page doesn't show up in a few seconds please click here.

By David Sierowski

[MOD] Placeholder v1.0.0 (no replies)

Sierowski — Sun, 13 May 2012 20:39:16 +0300

Sphider v1.3.5 Modification
---------------------------

Name And Version
----------------

[MOD] Placeholder v1.0.0

Description
-----------

This mod will add a placeholder to your search box. and be removed
when it loses focus.

01. replace the following code from search_form.html with the code from code1.txt

02. Add the code from code2.txt to the header file. Somewhere between AND

03. Copy placeholder.js to the include folder.

CODE1.TXT

CODE2.TXT

PLACEHOLDER.JS

function aph(elID) {
    if(document.getElementById(elID)) {
        var el = document.getElementById(elID);
		console.log(el);
		var pH = el.getAttribute('placeholder');
		console.log(el.getAttribute('placeholder'));
        if(pH !== 'undefined' && el.value == '') {
            el.value = pH;
            el.className += 'placeheld';
        }
    }
}
function cph(elID) {
    if(document.getElementById(elID)) {
        var el = document.getElementById(elID);
		console.log(el);
		var pH = el.getAttribute('placeholder');
        if(pH !== 'undefined' && el.value == pH) {
            el.value = '';
            el.className = el.className.replace('placeheld','');
        }
    }
}
var myElIDs = ['myinput','myinput2'];
console.log(myElIDs);
for(var i in myElIDs) {
    if(document.getElementById(myElIDs)) {
		console.log(document.getElementById(myElIDs));
		aph(myElIDs);
    }
}

By David Sierowski

Spell Check (no replies)

Sierowski — Fri, 11 May 2012 01:35:13 +0300

dose anybody have any ideas on adding spell check functionality.

Controlling Search Regardless of Input (no replies)

selym — Mon, 30 Apr 2012 22:01:31 +0300

I don't really expect a complete answer, but just throwing this out there in case there are any php/sphider savants out there.

I was just wondering if it's possible to control the search string regardless of what people input (basically disable the front end search functionality unbeknownst to user). This would be for a joke type search. For example, someone searches for "windows 7 troubleshooting" and get results for some other random (embarrassing, funny, etc.) search term that I have predetermined.

It would be funny to have a couple hundred search terms that were randomly selected from a pool of choices whenever someone used the search. What do you think? Would this be a ton of programming/work, or something rather simple to someone who's got a little game?