Web Crawling

Law Leecher is a multi-threaded web crawling tool which extracts laws from the EU law database PreLex (http://ec.europa.eu/prelex/). It's written in Ruby. The latest version is 1.4 You can also visit the SourceForge project management homepage. You will most probably find the version history...

Auto Web Browser saves a lot of time when you are looking for something on the Internet. No need for numerous clicking, waiting for pages download and closing annoying pop-ups. Just enter the words describing what you want to find. The Auto Web Browser will consult the search engines from a user...

This useful utility examines websites for errors and other issues using a built in web crawling engine. Errors checked for include broken or moved hyperlinks, missing page titles and missing meta tags. It also generates HTML for use in creating website site maps and XML for Google SiteMaps....


Platforms: Windows

The web crawler is a program that automatically traverses the web by downloading the pages and following the links from page to page. A general purpose of web crawler is to download any web page that can be accessed through the links. This process is called web crawling or spidering. Many sites,...


Platforms: Windows, Mac, Linux, Linux Console, Linux Gnome, Linux GPL, Linux Open Source, Java

Auto Web Browser saves a lot of time when you are looking for something on the Internet. No need for numerous clicking, waiting for pages download and closing annoying pop-ups. Just enter the words describing what you want to find. The Auto Web Browser will consult the search engines from a user...


Platforms: Windows

Zeus is an easy to use Intelligent, Internet Robot, that creates traffic to your web site, in numbers beyond your wildest dreams. Zeus creates thousands of link trades by automatically finding web sites you train him to find and then doing 95% of the work involved in Link Marketing. Zeus is...


Platforms: Windows, Windows CE

Data Toolbar is an Internet Explorer add-on to collect catalog style information from the web. The add-on converts structured web data into a table style format that can be loaded into a spreadsheet or a database. Based on powerful pattern recognition logic it automatically extracts thousands of...


Platforms: Windows, Other

As the name implies it automatically create a SiteMap of your web page. People love it and It also can do a lot more. For instance SiteMap can: * Create a 'siteindex.xml' file for to help google index better. * Validate that your web page doesn't have any nasty defaults in the title, filename...


Platforms: Mac

Octoparse is a powerful but easy-to-use and free web scraping tool that is being used by tens of thousands of individuals and companies all over the world. The word of Octoparse is the combination of the two words, “Octopus” and “Parse”....


Platforms: Windows, Windows 7

Trellian SiteSpider is a powerful, yet simple to use program which is loaded with sophisticated search, web crawling and site mapping functions. What does SiteSpider do? SiteSpider can be used simply as a web browser, which works cooperatively with Internet Explorer to allow importing and...


Platforms: Windows

Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing. News 2011-01-02 Scrapy 0.12 released! 2010-09-29 Scrapy...


Platforms: Mac

Zeus is the original Internet Robot. Zeus creates web site traffic, almost immediately, in numbers beyond your wildest dreams. Zeus creates thousands of link trades by automatically finding web sites that you train him to find. then Zeus does 95% of the work of Link Marketing. Zeus is the only...


Platforms: Windows

Lextek Language Identifier is capable of automatically identifying that language that a document is written in. Supporting over 260 different languages and encoding modules, the Lextek Language Identifier is capable of recognizing more languages and more character encodings than any other...


Platforms: Windows

Web Automator is a powerful tool for extracting content from Web sites and testing them. It allows you to create projects (like programs) - e.g. project for automated creating account. Each project has some variables connected to it. There are variables of two types: pre-defined and user-defined....


Platforms: Windows

Visual Web Spider is a software for crawling the web. It is a web crawler. It helps you to develop search directory for responsible website promotion, or for research purposes. Visual Web Spider builds targeted URL database at high speed. Our program allows you to extract URLs and metadata...


Platforms: Windows

Crawl the web with this free ActiveX web bot component. Advanced features include caching, "avoid" patterns, and robots.txt compliance.


Platforms: Windows

Dowser is a research tool for the web. It clusters results from major search engines, associates words that appear in previous searches, and keeps a local cache of all the results you click on in a searchable database. It helps you keep track of what you find on the web. Future versions will...


Platforms: *nix

SiteMapOrganizer is a powerful web site map generating and processing tool that fully simplifies processes of crawling web site in any accessible form (HTTP, FTP, Local on Disk), processing web site map to fully setting all the parameters of the site map up and generating the web site map in form...


Platforms: Windows, Windows 7

Web Accounts Plus is a great utility that allows users to maintain their ever growing list of web accounts, credit cards, and frequent traveler information for free ! The information can be password protected and the application can be hidden in the system tray. You can drag and drop...


Platforms: Windows

TracePlus/Web Detective (eBusiness Edition) is a performance analysis and diagnostic tool specifically designed for Web development. Analyze the HTTP and HTTPS protocols used by an application and provides performance and diagnostic information. Understands SOAP, WebDAV, DASL, and Delta-V...


Platforms: BeOS, Palm OS, Windows