American Funnel Web Spider
Spider websites to find slow or broken links, view SEO meta-data and measure performance.
SiteScan is a Windows desktop utility to spider websites for slow and broken links (including the HTTP response 404 not found error). It quickly crawls and validates all web pages, images, file...
Platforms: Windows 8, Windows 7, Windows Server
License: Shareware | Cost: $49.00 USD | Size: 13.31 MB | Download (49): SiteScan Broken Link Checker Download |
Aggregate, manipulate, and mashup data in one simple tool.
Djuggler (short for data juggler) is a hands-on visual scripting tool to mashup and collect data from multiple sources like web pages, text files, data grids as Microsoft Excel or CSV files. When the data has been collected or...
Platforms: Windows
License: Shareware | Cost: $319.00 USD | Size: 3.01 MB | Download (95): Djuggler Download |
This superb program allows you to automatically download pictures, movies, mp3 and other files to your hard drive for offline viewing. Itls a highly configurable, multi-threaded, file-retrieving Web spider. It's perfect for automated researching and downloading files from Web sites. It...
Platforms: Windows
License: Shareware | Cost: $24.95 USD | Size: 1.86 MB | Download (94): Internet Spider Download |
"Blindingly fast" (Computer Forensics:Incident Response Essentials) searching of gigabytes of word processor, database, spreadsheet, ZIP, XML, PDF, HTML, emails and more. *Over two dozen indexed, unindexed, fielded and full-text search options. *Highlights hits in retrieved files; for HTML & PDF,...
Platforms: Windows
License: Commercial | Cost: $199.00 USD | Size: 24 MB | Download (102): dtSearch Desktop with Spider Download |
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix (sometimes spelled heretrix, or misspelled or missaid as heratrix/heritix/ heretix/heratix) is an archaic word for heiress (woman who inherits). Since this crawler seeks to...
Platforms: Windows, Mac, *nix, Java, BSD Solaris
License: Freeware | Download (61): Heritrix Download |
Larbin is a Web crawler intended to fetch a large number of Web pages. It should be able to fetch more than 100 millions pages on a standard PC with much u/d. This set of PHP and Perl scripts, called webtools4larbin, can handle the output of Larbin.
Platforms: Windows, Mac, *nix, PHP, BSD Solaris
License: Freeware | Download (59): Webtools 4 larbin Download |
Fast, detailed website analysis for web-masters and content managers. Find slow, broken or mistyped links. Gather page and file meta-data. Find referrers and server redirects. Extract email addresses. Measure response and browser load-time. Save and export results.
Check any type of web page,...
Platforms: Windows, Windows 8, Windows 7, Windows Server
License: Shareware | Cost: $195.00 USD | Size: 13.77 MB | Download (54): SiteScan Download |
InterroBot combines a web crawler with advanced filtering to help you get answers to just what the heck is going on with your website.
Whether you're looking for a snippet of text in the body of your target webpages, the HTTP headers, or a specific HTTP status code-InterroBot will help you...
Platforms: Windows, Windows 8, Windows 7, Windows Server
License: Shareware | Cost: $65.00 USD | Size: 77.02 MB | Download (73): InterroBot Download |
WWW::PkgFind watches Web sites, FTP sites, GIT repositories, etc. for new code releases, and downloads them. In other words, its like a Web spider tuned for downloading software packages and patches. WWW::PkgFind is also able to generate a queue of incoming packages, to allow subsequent...
Platforms: *nix
License: Freeware | Size: 13.31 KB | Download (122): WWW::PkgFind Download |
Visual SEO Studio is a desktop SEO Audit Tool, a Visual approach to make SEO tasks quick and easy.
The program lets you Crawl one or more sites in parallel with its Interactive Web Spider, Spot on-site issues at a glance, Save Page Screenshot miniatures and see where the fold lays, visually...
Platforms: Windows, Windows 7, Windows Server
License: Freeware | Size: 9.83 MB | Download (447): Visual SEO Studio Download |
Webskape Browser is a web browser for public use that can be locked down to make browsing the internet secure and the operating system tamperproof. Great for cyber cafes, libraries, malls, airports, shops, museums, art galleries and more...
Internet Explorer
Webskape Browser uses the same...
Platforms: Windows
License: Shareware | Cost: $16.33 USD | Size: 1.73 MB | Download (56): Webskape Browser Download |
MailsExtractor is an easy-to-use and fast email collection tool. It works like any website spider. It can open all the pages from a site, and it will collect all the emails it finds. You should only setup a task with serval options at first, and then you can start or pause it whenever you want....
Platforms: Windows
License: Demo | Cost: $29.00 USD | Size: 1.3 MB | Download (52): MailsExtractor Download |
About NetScrape
If you like images on the web, you ll like NetScrape. It is a page-sucking web spider, surgically attached to an "aqua-licious" image thumbnailer and viewer. In this regard, it s a lot like Frankenstein - and he s pretty cool, right? Give NetScrape a URL, and it will "scrape" all...
Platforms: Mac
License: Shareware | Cost: $0.00 USD | Size: 768 KB | Download (49): NetScrape 1.6.2 Download |
Astanda Directory Project [ADP] is a directory, search engine, and spider/crawler software. Your own breed of DMOZ and Google.ADP is the only software in its class ever to be written in PHP with MySQL database support. Supports virtually unlimited number of links and pages without any loss of...
Platforms: PHP
License: Shareware | Cost: $0.00 USD | Size: 184.32 KB | Download (45): Astanda Open Directory Project Download |
Pattern is a web mining module for the Python programming language. It bundles tools for data retrieval (Google + Twitter + Wikipedia API, web spider, HTML DOM parser), text analysis (rule-based shallow parser, WordNet interface, syntactical + semantical n-gram search algorithm, tf-idf + cosine...
Platforms: *nix
License: Freeware | Size: 11.59 MB | Download (46): Pattern for Linux Download |
SiteAnalyzer it is a web crawler to scan the websites and check their technical and of SEO-parameters for errors and correct them effectively.
Key features of the SiteAnalyzer
- Scanning of all pages of the site, as well as images, scripts and documents
- Getting server response codes for...
Platforms: Windows, Windows 8, Windows 7, Windows Server
License: Freeware | Size: 8.13 MB | Download (737): SiteAnalyzer Download |
Visual SEO Studio is a desktop SEO Audit Tool, a Visual approach to make SEO tasks quick and easy.
The program lets you Crawl one or more sites in parallel with its Interactive Web Spider, Spot on-site issues at a glance, Save Page Screenshot miniatures and see where the fold lays, visually edit...
Platforms: Mac
License: Free | Cost: $0.00 USD | Size: 47.59 MB | Download (25): Visual SEO Studio for Mac Download |
SXBandMaster is designed for Internet services maximum use. It is a program complex that serves as ISP Dialer, Download Manager, Offline Browser (Web Spider), FTP Upload Application with FXP support, Time Synchronizer (Atomic Clock), Traffic Explorer and many others. SXBandMaster excels it's...
Platforms: Windows
License: Shareware | Cost: $9.95 USD | Size: 3.1 MB | Download (96): SXBandMaster Download |
SXBandMaster is designed for Internet services maximum use. It is a program complex that serves as ISP Dialer, Download Manager, Offline Browser (Web Spider), FTP Upload Application with FXP support, Time Synchronizer (Atomic Clock), Traffic Explorer and many others. SXBandMaster excels it's...
Platforms: Windows
License: Shareware | Cost: $9.95 USD | Size: 3.28 KB | Download (91): SXBandMaster 0.94 build Download |
PHP FastFileCache caches output from dynamic PHP scripts, and stores them in files for fast retrieval under high server load.. WebSuck is a web spider for Java2 (C) 2001 Åke Wallebom
Platforms: *nix
License: Freeware | Size: 2.05 KB | Download (97): PHP FastFileCache Download |