Scraping

WWW::PDAScraper is a Perl class for scraping PDA-friendly content from websites. Synopsis use WWW::PDAScraper; my $scraper = WWW::PDAScraper->new qw ( NewScientist Yahoo::Entertainment ); $scraper->scrape(); or use WWW::PDAScraper; my $scraper = WWW::PDAScraper->new; $scraper->scrape...

Diggernaut is a cloud based service for web scraping, data extraction and other ETL (Extract, Transform, Load) tasks. If you are a reseller of goods and your supplier does not let you have their data in a suitable format, such as excel or csv, you will have to manually retrieve data from their...

FEAR::API is a tool that helps reduce the time you spend creating site scraping scripts and helps you do it in a much more elegant way. FEAR::API combines many strong and powerful features from various CPAN modules, such as LWP::UserAgent, Template::Extract, WWW::Mechanize, Encode,...


Platforms: *nix

If you love the Chrome web browser, but are tired of repetitive tasks like visiting the same sites every days, filling out forms, and remembering passwords, then iMacros for Chrome is the solution you've been dreaming of! iMacros was designed to automate the most repetitious tasks on the...


Platforms: Windows, Mac, *nix

Makes scraping sites, parsing XML, and following hyperlinks a snap. A number of other scraper-type modules depend on it, so its easier to update one place than in many. Saves developers a lot of time.The module can be extended to include features such as proxy rotation, delayed hits or useragent...


Platforms: Windows, Mac, *nix, PHP, BSD Solaris

A Python library for scraping things. Features include: * HTTP, HTTPS, FTP requests via an identical API * HTTP caching, compression and cookies * redirect following * request throttling * robots.txt compliance (optional) * robust error handling scrapelib is a project of Sunlight Labs (c)...


Platforms: *nix

ScraperPOD is a framework for scraping results from search engines. SYNOPSIS use WWW::Scraper; # Name your Scraper module / search engine as the first parameter, use WWW::Scraper(eBay); # or in the new() method $scraper = new WWW::Scraper(eBay); Classic WWW::Search mode # Use a...


Platforms: *nix

IRobot is the Internet-robot engine for Visual Web automation and Web scraping. Completely automatic Web recording. Create your own IRobot Web robot agents to automate everything on the Web. Support Web data integration and computation. Share your Web experience with your friends. Batch form...


Platforms: Windows

YACY is a p2p-based search engine for the WWW, based on it's distributed crawler and the built-in caching/scraping http proxy. Brings it's own database, has a httpd/web interface. Implemented in Java, supports all platforms.


Platforms: Windows, Mac, Linux

This is a simple yet powerful PHP class for screen scraping.It uses PHP CURL Library and handles most of the work for you. It has several useful functions for cleaning and extracting the needed content.The script is compatible with all operating systems and is very easy to install and use.


Platforms: PHP

This project sets out to create a powerful, robust, quick to deploy, web scraper. A web scraper is a tool that extracts specific parts of web pages rather than the entire html page as a crawler would. The outcome of this project will be a tool, written in Java, that accomplishes the following...


Platforms: Mac

galerts is a Python client for managing Google Alerts. It (currently) resorts to scraping HTML from Google's web interface since there is no public API (yet). Please find galerts on github if you would like to collaborate. md5=69a793aba56a8edc063846088e958631


Platforms: *nix

Pexpect is a pure Python module that makes Python a better tool for controlling and automating other programs. Pexpect is similar to the Don Libes `Expect` system, but Pexpect as a different interface that is easier to understand. Pexpect is basically a pattern matching system. It runs programs...


Platforms: *nix

roscraco (short for ROuter SCRAping and COntrol) is a library for managing networking devices (routers). The targeted (and currently supported) devices are home routers. Settings are fetched and pushed from/to devices by using their standard web-management interface (HTTP). The type of device...


Platforms: *nix

Automate manual task of scraping search results URL from Google for PPV and other site promotion tools. Supports multiple Google searches, proxy servers. Software is free and easy to use. Make all your Internet promotion tasks much easier and more automatic using this software to search for...


Platforms: Windows, Windows 8, Windows 7, Windows Server

Octoparse is a powerful but easy-to-use and free web scraping tool that is being used by tens of thousands of individuals and companies all over the world. The word of Octoparse is the combination of the two words, “Octopus” and “Parse”....


Platforms: Windows, Windows 7

Add encrypted and protected images to WordPress web pages. Upload and add encrypted images from the WP editor. Secure Image encrypted images provide protection for images from page saves and web scraping. Images are safe from download and saving while on display in a web browser and also safe...


Platforms: Windows, Windows 8, Windows 7, Windows Server

Use the ActiveX controls included with the PowerTCP Emulation Tool to build advanced terminal emulation applications (VT-52, VT-100, VT-220, and VT-320 supported). Other supported functionality includes screen scraping, advanced graphic fonts, superior printing capabilities, and support for...


Platforms: Windows

Use the ActiveX controls included with the PowerTCP Web Tool to communicate with web servers using HTTP or HTTPS. Perfect for web site scraping, automating form submission, or integrating HTTP into a Windows application using the HTTP GET or PUT command. Other features include automatic or...


Platforms: Windows

The Web browser is probably the most frequently used software today, and many tasks are highly repetitious. iMacros relieves the repetition of checking on the same sites every day, remembering passwords and filling out web forms. Web professional can use the software for functional testing and...


Platforms: Windows