Crawler
Astanda Directory Project [ADP] is a directory, search engine, and spider/crawler software. Your own little breed of DMOZ and Google. ADP is the only software in its class ever to be written in PHP with MySQL database support. Supports virtually unlimited ammount of links and pages. Powered by...
Platforms: Windows, Mac, *nix, PHP, BSD Solaris
License: Freeware | Download (97): Astanda Directory Project 1.3b Download |
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix (sometimes spelled heretrix, or misspelled or missaid as heratrix/heritix/ heretix/heratix) is an archaic word for heiress (woman who inherits). Since this crawler seeks to...
Platforms: Windows, Mac, *nix, Java, BSD Solaris
License: Freeware | Download (55): Heritrix Download |
Larbin is a Web crawler intended to fetch a large number of Web pages. It should be able to fetch more than 100 millions pages on a standard PC with much u/d. This set of PHP and Perl scripts, called webtools4larbin, can handle the output of Larbin.
Platforms: Windows, Mac, *nix, PHP, BSD Solaris
License: Freeware | Download (59): Webtools 4 larbin Download |
PRO-Search is a crawler of FTP servers, SMB shares, HTTP, dc networks with powerful web search and navigation interface.
Platforms: Windows, Mac, *nix, Perl, BSD Solaris
License: Freeware | Download (68): PRO-Search Download |
PKTorrents has the functionality to crawl the top torrent sites, Meganova, Mininova, Piratebay, Snarf, Torrentportal, Torrentspy and adding 1000's of torrents to the db as it gos using the well know ibitzy torrent spider/crawler!! Users can now update torrents, rate torrents and use simple BB...
Platforms: Windows, Mac, *nix, PHP, BSD Solaris
License: Freeware | Download (57): PKTorrents 0.1b Download |
Bee-rain is a web crawler that harvest and index file over the network. It is a free PHP search engine software.Bee-rain helps you in your sleep by comparing, filtering, analyzing citations of brands, companies and personalities on the Internet. Bee-rain then produces statistics and visualization...
Platforms: PHP
License: Freeware | Size: 4.28 MB | Download (46): Bee-rain Download |
FoxySpider is your personal web crawler! It can crawl into a given web site and retrieve what you really want (video clips, images, music files, etc.). FoxySpider displays the located items in a well-structured thumbnail gallery for ease of use.
Platforms: Mac
License: Freeware | Size: 71.68 KB | Download (34): FoxySpider Download |
AWSpider is a web crawler for Amazon Web Services, written in Python. #md5=53f2f7be2f926c9f4b2c867b657ef59b
Platforms: Mac
License: Freeware | Size: 40.96 KB | Download (36): AWSpider Download |
Bitvo is a media crawler and downloader. Point it to a web site or thumbnail gallery and it will seek out images, videos, and audio files and download them directly to your computer.
Platforms: Mac
License: Shareware | Cost: $0.00 USD | Size: 10.24 KB | Download (43): Bitvo Download |
AWSpider is a web crawler for Amazon Web Services, written in Python. #md5=edb1228e36df1a72e28bb07119e7a2e2
Platforms: *nix
License: Freeware | Size: 40.96 KB | Download (35): John Wehr Download |
AWSpider is a web crawler for Amazon Web Services, written in Python #md5=edb1228e36df1a72e28bb07119e7a2e2
Platforms: *nix
License: Freeware | Size: 40.96 KB | Download (34): AWSpider for Linux Download |
Arclab Website Link Analyzer is a software website crawler to analyze and optimize your website. It spiders your whole website checking for broken links, duplicate content, missing tags and other SEO issues.
The program spiders your website the same way as search engine robots. It scans each...
Platforms: Windows, Windows 8, Windows 7, Windows Server
License: Shareware | Cost: $49.95 USD | Size: 3.95 MB | Download (89): Arclab Website Link Analyzer Download |
InterroBot combines a web crawler with advanced filtering to help you get answers to just what the heck is going on with your website.
Whether you're looking for a snippet of text in the body of your target webpages, the HTTP headers, or a specific HTTP status code-InterroBot will help you...
Platforms: Windows, Windows 8, Windows 7, Windows Server
License: Shareware | Cost: $65.00 USD | Size: 77.02 MB | Download (67): InterroBot Download |
Provides complete stock option chains from Internet, calculates all standard option values including Theoretical Value, Percent to Double, Implied Volatility, the Greeks, Historical Volatility. Allows for easy building of option strategy by pointing and clicking on individual contracts from the...
Platforms: Windows
License: Shareware | Cost: $99.95 USD | Size: 11.75 MB | Download (265): Option Crawler Download |
NewzCrawler is a web news reader & browser which provides access to the a news content from different sources: - Syndicated news given in XML-based formats (RSS, ATOM, CDF, BackSlash, ScriptingNews etc.); - Usenet (NNTP) newsgroups; - Web pages. NewzCrawler gathers news content from each source...
Platforms: Windows
License: Shareware | Cost: $24.95 USD | Size: 1.37 MB | Download (238): Newz Crawler Download |
GluedOn keeps your computer on the Internet by simulating activity. This keeps your dial-up ISP from terminating your connection due to inactivity. GluedOn is a crawler, and includes many options. Proxy server support is included. GluedOn can be minimized to the system tray for easy access. You...
Platforms: Windows
License: Shareware | Cost: $9.99 USD | Size: 883 KB | Download (139): GluedOn Download |
There are millions of guest books at the web! Imagine, that only a little part of them have your link. The software enables you to post to many guest books. You are searching in search engines for guest books and the software is submitting the link. You must care for the submission of every link,...
Platforms: Windows
License: Shareware | Cost: $149.00 USD | Size: 396 KB | Download (102): Guest book Crawler Download |
Astanda Directory Project is a directory, search engine, and spider/crawler software. Your own little breed of DMOZ and Google. ADP is the only software in its class ever to be written in PHP with MySQL database support. Supports virtually unlimited ammount of links and pages. Powered by Smarty...
Platforms: Windows
License: Shareware | Cost: $0.00 USD | Size: 181 KB | Download (595): ADP Download |
WebReaper is web crawler or spider, which can work its way through a website, downloading pages, pictures and objects that it finds so that they can be viewed locally, without needing to be connected to the Internet. The sites can be saved locally as a fully-browsable website which can be viewed...
Platforms: Windows
License: Freeware | Size: 1.2 MB | Download (254): WebReaper Download |
Spiderman has been in many films, comics and adventures. By day Peter Parker, a young science student, Spiderman has millions of fans across the world, and he now actually forms a part of a new breed of superheroes called The New Avengers.If you think you are a fan of Spiderman, you can now...
Platforms: Windows
License: Freeware | Size: 1024 B | Download (222): Spiderman Wall Crawler Download |