Watermelon Crawl Tracy Byrd

As more and more Web sites have become dynamically generated (with dynamic URL's), search engines have improved in order to crawl and index more of these dynamic sites. Search engines such as Google and Yahoo have been actively crawling and indexing dynamic URL's, such as those found with...

Sitemap Creator is a php script making use of the announced new standard SiteMaps protocol supported by Google, Yahoo and MSN and pings them to crawl it. It also sends an email to you with the crawler info and which sitemap was crawled.

PKTorrents has the functionality to crawl the top torrent sites, Meganova, Mininova, Piratebay, Snarf, Torrentportal, Torrentspy and adding 1000's of torrents to the db as it gos using the well know ibitzy torrent spider/crawler!! Users can now update torrents, rate torrents and use simple BB...


Platforms: Windows, Mac, *nix, PHP, BSD Solaris

This module is under active development for a Drupal 6 upgrade have a look at the roadmap/progress Nutch RoadmapNutch is a web crawler/indexer/search engine that is based on Lucene. It is a Java tool.This module allows you to have basic control over the Nutch crawl lifecycle through the Drupal...


Platforms: PHP

Most often Google AdSense place ads on your site that are relevant to your content but sometimes Google gets it wrong. It can also be times where you don't want Google to put any relevance to a certain text on your site when they crawl it.An example would be if you write a blog post about that...


Platforms: PHP

PK-Torrents is a PHP torrent lister based on torrenthoster v1.0. It has the functionality to crawl the top torrent sites, Meganova, Mininova, Piratebay, Snarf, Torrentportal, Torrentspy.


Platforms: PHP

Las Vegas 2 - by Kate England Let it all ride with more icons from the casino! Keywords: all bell bet betting black blackjack cards cash casino chance cherry club diamond england fruit gamble heart iconfactory icons jack jackpot kate las las2 lemon money orange osx plum poker ride seven slots...


Platforms: Mac

FoxySpider is your personal web crawler! It can crawl into a given web site and retrieve what you really want (video clips, images, music files, etc.). FoxySpider displays the located items in a well-structured thumbnail gallery for ease of use.


Platforms: Mac

The Web Archive eXtensions (WAX) include adaptation of the Nutch fetcher step to go against web archives rather than crawl the open net - adaptation currently does Internet Archive ARC files only - and plugins to add extra fields to the index that return an Archive Records' location in the...


Platforms: Mac

sqlsus is an open source MySQL injection and takeover tool, written in perl. Via a command line interface, you can retrieve the database(s) structure, inject your own SQL queries (even complex ones), download files from the web server, crawl the website for writable directories, upload and...


Platforms: *nix

Lips of Suna is a tongue-in-cheek dungeon crawl game that takes place in the chaotic dungeons of Suna. The five intelligent races of the world descend to the dungeons with their goal to save the world from a conclusive disaster. In your journey to the depths of the dungeons, you will, among...


Platforms: *nix

Crawlzilla is a cluster-based search engine deployment tools. It helps user to build search engine in your cluster, and offers management mechanism (such as: cluster management, crawl management, index pool management...).


Platforms: *nix

Clickie is a handy utility that will add 2 new features to the context menu (right click) of Windows Explorer: 1) Copy Path Name: Copies the path of the selected folder or file to the clipboard (as text). You can then paste it wherever you need it. Some softwares need the name and path of a file...

ASP-DOC will spider all the related files in the system and list all the functions for each file. This will help dramatically, in the documentation process. It will read- in included files and normal links that are relative to the "default" page where you start the crawl. It will simply display a...


Platforms: Windows

Creepy crawly screensaver. A horde of maggots crawl all over your screen eating away at it and revealing what's underneath. You can choose to run it on a black screen or let the maggots have a go at your desktop. New in v2001 are the turbo speed option, that makes the maggots go like...


Platforms: Windows

Easy CD Insert and Eject is a software created by SenJen, which lets you open and close your CD tray without spending any energy. After installing this program, it is best if you look for its route in Windows Explorer, and create a direct access on your desktop. In this way, to open or close the...


Platforms: Windows

Enjoy all the stars of the NBA on your screen with this fantastic NBA All Star Game 2006 wallpaper. In this wallpaper, with 1024x768 pixel resolution, you can see the ten players that form two initial quintets: Allen Iverson, Dwayne Wade, LeBron James, Jermaine O+Neal and Shaquille O+Neal for the...


Platforms: Windows

Sharehound is a file systems indexer and Google-like searcher. Sharehound project currently supports SMB file shares (i.e. MS Windows-based network shares are supported). The Web UI is used for search and crawl monitoring. Files and directories are indexed and searched by their paths and some...


Platforms: *nix

ht://Check is more than a link checker. Its a console application written for GNU/Linux systems in C++ and derived from the best search engine available on the Internet for free: ht://Dig. It is very useful for Webmasters who want to control and monitor their Websites, in order to discover...


Platforms: *nix

rdiffWeb is designed to operate with two different backends: MySQL and configuration file. Restoring folders is supported, as is restoring older revisions of files. An RSS feed is available for backup status. File backend The simplest mode uses a configuration file-based backend. This mode...


Platforms: *nix