Crawl

Crawl-By-Example runs a crawl, which classifies the processed pages by subjects and finds the best pages according to examples provided by the operator. Crawl-By-Example is a plugin to the Heritrix crawler, and was done as a part of GSoC06 program. Platforms: Windows, Mac, Linux

Dungeon Crawl Stone Soup is a free rogue-like game of exploration and treasure-hunting. Stone Soup is a continuation of Linley's Dungeon Crawl. It is openly developed and invites participation from the Crawl community. See http://crawl.develz.org ! Platforms: Windows, Mac, BSD, Linux

Dungeon Crawl Stone Soup is a free roguelike game of exploration and treasure-hunting in dungeons filled with dangerous and unfriendly monsters in a quest for the mystifyingly fabulous Orb of Zot. Dungeon Crawl Stone Soup has diverse species and many different character backgrounds to choose... Platforms: Mac

ZhuaShuShell project is a set of bash shell scripts to crawl online e-books (in HTML format) from certain Chinese e-book sites and save the data that is formatted as a single text book to your local machine. Platforms: *nix

Displays an RSS feed in the Star Wars "Crawl" style. Platforms: Mac

Niocchi is a java asynchronous crawl library implemented with NIO. It is designed to crawl several thousands of hosts in parallel on a single low end server. It is currently being used in production by Enormo to crawl thousands of websites daily, and by Vitalprix. Platforms: Mac

Creepy crawly screensaver. A horde of maggots crawl all over your screen eating away at it and revealing what&aposs underneath.You can choose to run it on a black screen or let the maggots have a go at your desktop.New in v2001 are then turbo speed option, that makes the maggots go like piranhas...

Creepy crawly screensaver, Maggots crawl all over your screen eating away at it and revealing what's underneath. You can choose to have them feed on a black screen or your desktop, the munching sounds make it even more gross, & there's a speed option. Switch the saver to turbo to see the maggots... Platforms: Windows

Visual SEO Studio is a desktop SEO Audit Tool, a Visual approach to make SEO tasks quick and easy. The program lets you Crawl one or more sites in parallel with its Interactive Web Spider, Spot on-site issues at a glance, Save Page Screenshot miniatures and see where the fold lays, visually... Platforms: Windows, Windows 7, Windows Server

Crawl the web with this free ActiveX web bot component. Advanced features include caching, "avoid" patterns, and robots.txt compliance. Platforms: Windows

Crawl a website with this free ActiveX spidering component. Advanced features include caching, "avoid" patterns, and robots.txt compliance. Platforms: Windows

Inspired by the tickers of 24-hour cable news, Tickershock is an RSS, RDF and Atom news reader that displays news feeds anywhere on your desktop in a animated crawl or billboard display. - Added new icon and Spaces support for Leopard. - Improved HTML entity parsing in ticker titles. - Fixed... Platforms: Mac

Power Search can find or capture anything embedded in HTML, CSS or JavaScript. Many websites are built from server-side scripts that dynamically generate content each time a page is loaded. Website content may come from databases, files, or from a different server all together. Power Search... Platforms: Windows, 2003, Windows Vista, 2008, 7, 7x64

XenoCrawler is an application that allows you to crawl websites, servers, and web pages to view source code, style sheets, scripts, html, graphics, documents, and other related information safely, quickly, and easily without opening a browser or running any scripts. XenoCrawler is targeted for... Platforms: Windows, Other

Eveyone dreaming abaout their site will rank in search angine and for that they dream that they website is search engine optimized, but for reach that they have to pay and the cost is so expensive, so because that i create this ultimate wordpress seo, you know wordpress is blog script with SEO... Platforms: Windows, Edition 2005,Edition 2005

FTPSeek is an opensource (GPL) script which has the capability to crawl multiple FTP servers, allowing you to instantly search multiple servers at once, and browse through the file structures of each server at your will. Platforms: Windows, Mac, Linux

As more and more Web sites have become dynamically generated (with dynamic URL's), search engines have improved in order to crawl and index more of these dynamic sites. Search engines such as Google and Yahoo have been actively crawling and indexing dynamic URL's, such as those found with... Platforms: Windows, *nix, PHP, BSD

Sitemap Creator is a php script making use of the announced new standard SiteMaps protocol supported by Google, Yahoo and MSN and pings them to crawl it. It also sends an email to you with the crawler info and which sitemap was crawled. Platforms: Windows, Mac, *nix, PHP, BSD Solaris

PKTorrents has the functionality to crawl the top torrent sites, Meganova, Mininova, Piratebay, Snarf, Torrentportal, Torrentspy and adding 1000's of torrents to the db as it gos using the well know ibitzy torrent spider/crawler!! Users can now update torrents, rate torrents and use simple BB... Platforms: Windows, Mac, *nix, PHP, BSD Solaris

This module is under active development for a Drupal 6 upgrade have a look at the roadmap/progress Nutch RoadmapNutch is a web crawler/indexer/search engine that is based on Lucene. It is a Java tool.This module allows you to have basic control over the Nutch crawl lifecycle through the Drupal... Platforms: PHP