Crawler

The CMS - Bandits is a set of PHP scripts that implement an online HTML editor, calendar, search engine, RSS reader and editor, image gallery, comment system,Web crawler and many more.

Log RSS Creator allows you to watch live how users enter and leave your site.What information the Log RSS Creator shows :- title entry of the feed is the host name;- description entry is the list of pages visited by this host;- link entry is the link to the summary view for this host: which pages...

This module ties into Feeds and the XPath Parser modules, allowing you to scrape a site's old content. You simply provide an XPath query for the next button on the page.Currently usable, yay!


Platforms: PHP

The GeoBlocker module allows site maintainers to black-list or to white-list access to a Drupal site by countries. IP tables come from http://www.php.net/source.php?url=/include/ip-to-country.inc and are downloaded locally to speed up lookup.Features * Supports blacklisting as well white-listing;...


Platforms: PHP

Marmoset's purpose is to provide a simple way to make your Web site's contents available through intelligent search.This module integrates the Calais Marmoset site crawler into Drupal. It adds a tab on the node that when selected will display the microformat data returned my marmoset.Click...


Platforms: PHP

PK-Torrents is a PHP torrent lister based on torrenthoster v1.0. It has the functionality to crawl the top torrent sites, Meganova, Mininova, Piratebay, Snarf, Torrentportal, Torrentspy.


Platforms: PHP

PHP Search is a search engine script that searches a MySQL database for links and descriptions much like google. Manual adding of Data. Crawler Coming soon! Demo at http://www.jhosting.tk/admin/search/search.php


Platforms: *nix

This simple app does validation on all the HTML generated by your Django app, and keeps track of any pages that fail. You will no longer have any excuse for pages that aren't valid HTML What it does It uses a middleware to intercept all outgoing HTML and validate it using an external command...


Platforms: *nix

SiteAnalyzer it is a web crawler to scan the websites and check their technical and of SEO-parameters for errors and correct them effectively. Key features of the SiteAnalyzer - Scanning of all pages of the site, as well as images, scripts and documents - Getting server response codes for...


Platforms: Windows, Windows 8, Windows 7, Windows Server

Tiny Search Engine searhes a range of Web sites for given keywords. Unlike other searchers it doesn&apost make use of on-line search engines and so offers more up-to-date search results. Features include embedded web server, content filters and other advanced tools.


Platforms: Windows

Free full version of Spyware Terminator includes Real-Time Protection, HIPS and antivirus. Effectively remove spyware, adware, trojans, keyloggers, home page hijackers and other malware threats - even dangerous threats like Look2Me, BetterInternet, VX2, and CWS. Spyware Terminator is easy to use,...


Platforms: Windows

The Zoom Search Engine is a free package that allows web developers to add an industrial-strength custom search to their website in a matter of minutes. It is an extremely easy-to-use application that requires no complicated server-side set-up procedures. It can also be used to create search...


Platforms: Windows

Web Security Guard alerts help prevent you from entering potentially dangerous websites that may cause adware, viruses, spyware or spam infection. Available for free for home and office use, it uses a proprietary system to classify websites ranging from safe to dangerous and draws upon a growing...


Platforms: Windows

.AudioLink is a tool that makes searching for music on your local storage media easier and faster. Your searches can include a variety of criteria, like male artists, female artists, band, genre, etc. It is flexible, you have options of using a command line interface, multiple choices of GUIs,...


Platforms: *nix

Wayback Machine is an open source java implementation of the The Internet Archive Wayback Machine. The current production version of the Wayback Machine is implemented in perl, and lacks in maintainability and extensibility. Also, the code is not open source. Primary motivation for the new...


Platforms: *nix

External Site Catalog allows you to index and search external sites in a Plone site. ExternalSiteCatalog is a web crawler that can index external sites and make them searchable in Plone. You can specify the sites to index in a Plone Configlet, and directly index them from Plone, or let a...


Platforms: *nix

Minalyzer Lite project index and search tools to your internet or intranet website. Minalyzer Lite supports user searches by indexing data from combinations of databases, file systems and websites. The project runs on almost all Operating Systems. It even copes with restrictive shared hosting....


Platforms: *nix

BitTorrent Queue Manager is a console-based BitTorrent client running on top of BitTornado that provides built-in queue management functions. BitTorrent Queue Manager also provides a remote interface compatible with ABC for Web-based control. Furthermore, peer information can be queried,...


Platforms: *nix

Apache2::ClickPath is an Apache WEB Server User Tracking. SYNOPSIS LoadModule perl_module ".../mod_perl.so" PerlLoadModule Apache2::ClickPath < ClickPathUAExceptions > Google Googlebot MSN msnbot Mirago HeinrichderMiragoRobot Yahoo Yahoo-MMCrawler Seekbot Seekbot Picsearch psbot...


Platforms: *nix

Desktop IRIS is an easy-to-use search program that can be successfully downloaded and accessed by anyone. It allows you to intuitively find stored information from your desktop and network without imposing any restrictions on the number of files and folder locations indexed. The system uses the...


Platforms: Windows, Mac