Copart Crawler

Astanda Directory Project [ADP] is a directory, search engine, and spider/crawler software. Your own little breed of DMOZ and Google. ADP is the only software in its class ever to be written in PHP with MySQL database support. Supports virtually unlimited ammount of links and pages. Powered by...

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix (sometimes spelled heretrix, or misspelled or missaid as heratrix/heritix/ heretix/heratix) is an archaic word for heiress (woman who inherits). Since this crawler seeks to...

Larbin is a Web crawler intended to fetch a large number of Web pages. It should be able to fetch more than 100 millions pages on a standard PC with much u/d. This set of PHP and Perl scripts, called webtools4larbin, can handle the output of Larbin.


Platforms: Windows, Mac, *nix, PHP, BSD Solaris

PRO-Search is a crawler of FTP servers, SMB shares, HTTP, dc networks with powerful web search and navigation interface.


Platforms: Windows, Mac, *nix, Perl, BSD Solaris

PKTorrents has the functionality to crawl the top torrent sites, Meganova, Mininova, Piratebay, Snarf, Torrentportal, Torrentspy and adding 1000's of torrents to the db as it gos using the well know ibitzy torrent spider/crawler!! Users can now update torrents, rate torrents and use simple BB...


Platforms: Windows, Mac, *nix, PHP, BSD Solaris

Bee-rain is a web crawler that harvest and index file over the network. It is a free PHP search engine software.Bee-rain helps you in your sleep by comparing, filtering, analyzing citations of brands, companies and personalities on the Internet. Bee-rain then produces statistics and visualization...


Platforms: PHP

FoxySpider is your personal web crawler! It can crawl into a given web site and retrieve what you really want (video clips, images, music files, etc.). FoxySpider displays the located items in a well-structured thumbnail gallery for ease of use.


Platforms: Mac

AWSpider is a web crawler for Amazon Web Services, written in Python. #md5=53f2f7be2f926c9f4b2c867b657ef59b


Platforms: Mac

Bitvo is a media crawler and downloader. Point it to a web site or thumbnail gallery and it will seek out images, videos, and audio files and download them directly to your computer.


Platforms: Mac

AWSpider is a web crawler for Amazon Web Services, written in Python. #md5=edb1228e36df1a72e28bb07119e7a2e2


Platforms: *nix

AWSpider is a web crawler for Amazon Web Services, written in Python #md5=edb1228e36df1a72e28bb07119e7a2e2


Platforms: *nix

Arclab Website Link Analyzer is a software website crawler to analyze and optimize your website. It spiders your whole website checking for broken links, duplicate content, missing tags and other SEO issues. The program spiders your website the same way as search engine robots. It scans each...


Platforms: Windows, Windows 8, Windows 7, Windows Server

InterroBot combines a web crawler with advanced filtering to help you get answers to just what the heck is going on with your website. Whether you're looking for a snippet of text in the body of your target webpages, the HTTP headers, or a specific HTTP status code-InterroBot will help you...


Platforms: Windows, Windows 8, Windows 7, Windows Server

Provides complete stock option chains from Internet, calculates all standard option values including Theoretical Value, Percent to Double, Implied Volatility, the Greeks, Historical Volatility. Allows for easy building of option strategy by pointing and clicking on individual contracts from the...


Platforms: Windows

NewzCrawler is a web news reader & browser which provides access to the a news content from different sources: - Syndicated news given in XML-based formats (RSS, ATOM, CDF, BackSlash, ScriptingNews etc.); - Usenet (NNTP) newsgroups; - Web pages. NewzCrawler gathers news content from each source...


Platforms: Windows

GluedOn keeps your computer on the Internet by simulating activity. This keeps your dial-up ISP from terminating your connection due to inactivity. GluedOn is a crawler, and includes many options. Proxy server support is included. GluedOn can be minimized to the system tray for easy access. You...


Platforms: Windows

There are millions of guest books at the web! Imagine, that only a little part of them have your link. The software enables you to post to many guest books. You are searching in search engines for guest books and the software is submitting the link. You must care for the submission of every link,...


Platforms: Windows

Astanda Directory Project is a directory, search engine, and spider/crawler software. Your own little breed of DMOZ and Google. ADP is the only software in its class ever to be written in PHP with MySQL database support. Supports virtually unlimited ammount of links and pages. Powered by Smarty...


Platforms: Windows

WebReaper is web crawler or spider, which can work its way through a website, downloading pages, pictures and objects that it finds so that they can be viewed locally, without needing to be connected to the Internet. The sites can be saved locally as a fully-browsable website which can be viewed...


Platforms: Windows

Spiderman has been in many films, comics and adventures. By day Peter Parker, a young science student, Spiderman has millions of fans across the world, and he now actually forms a part of a new breed of superheroes called The New Avengers.If you think you are a fan of Spiderman, you can now...


Platforms: Windows