Web Crawler Pic

Desktop application for crawling (spidering) the web and extracting words, phrases, email addresses, web addresses, and everything you want from the web pages.

Octoparse is a powerful but easy-to-use and free web scraping tool that is being used by tens of thousands of individuals and companies all over the world. The word of Octoparse is the combination of the two words, “Octopus” and “Parse”....

PHPCrawl is a set of classes written in PHP for crawling/spidering websites, so just call it a webcrawler-library for PHP. The crawler "spiders" websites and delivers information about all found pages, links, files and so on to users of the library. By overriding a special method of the...


Platforms: Windows, Mac, *nix, PHP, BSD Solaris

RapidBot is the incredibly easy tool designed to create a robots.txt file for your site directly from RapidWeaver. Robots.txt file is retrieved by search engines, like Google and Bing, and used to state what pages need indexing and what must be ignored. A robot (also known as Spider or Web...


Platforms: Mac

A search engine reports that many thousands of pages contain the keywords you just entered. That is fine, but the first returned pages are just better advertised and probably not of big interest. Especially if you are looking for the best sources of information. On the other hand, it is hardly...


Platforms: Windows

Wayback Machine is an open source java implementation of the The Internet Archive Wayback Machine. The current production version of the Wayback Machine is implemented in perl, and lacks in maintainability and extensibility. Also, the code is not open source. Primary motivation for the new...


Platforms: *nix

External Site Catalog allows you to index and search external sites in a Plone site. ExternalSiteCatalog is a web crawler that can index external sites and make them searchable in Plone. You can specify the sites to index in a Plone Configlet, and directly index them from Plone, or let a...


Platforms: *nix

Boolean Search is a Mac OS X server application for searching the contents of plain and HTMLformatted text, PDF, MS Word and RTF files. Boolean Search can be employed as a search engine for a public website on theInternet, or a personal or group owned collection of files on your own computer or...


Platforms: Mac

Desktop IRIS is an easy-to-use search program that can be successfully downloaded and accessed by anyone. It allows you to intuitively find stored information from your desktop and network without imposing any restrictions on the number of files and folder locations indexed. The system uses the...


Platforms: Windows, Mac

The Indexing Service Companion allows the Microsoft Windows Indexing Service to retrieve and index content from remote websites (including those running Apache/PHP on Linux/Unix) and ODBC databases. Includes sample code for ASP (VBScript) and .NET Framework (C# and VB.NET). The Indexing Service...


Platforms: Windows

Netifera is a new modular open source platform for creating network security tools. This project provides many advantages for both security developers and researchers who want to implement new tools as well as the community of users of these tools. * TCP and UDP network scanning and service...


Platforms: Mac, Linux

This plugin was born because there was a no other real alternative to enable content sparationd and some CMS like functionalities in WordPress. The main goal was, to enhance WordPressd-deOaos functionalities, to hide some unwanted categories, from defined parts of the blog.Today, ACE can override...


Platforms: PHP

This module is under active development for a Drupal 6 upgrade have a look at the roadmap/progress Nutch RoadmapNutch is a web crawler/indexer/search engine that is based on Lucene. It is a Java tool.This module allows you to have basic control over the Nutch crawl lifecycle through the Drupal...


Platforms: PHP

Xapian is an Open Source Probabilistic Information Retrieval library, released under the GPL. Xapian iss written in C , with bindings to allow use from other languages (Perl, Java, Python, PHP, and TCL are currently supported; Guile and C# are being worked on). Xapian is designed to be a highly...


Platforms: *nix

Email Extractor Online was created to help marketers, internet entrepreneurs and sales professionals around the globe gather the most important piece of information in modern day communication; the email address. In most cases this information can be found by visiting a URL, searching for the...


Platforms: Windows, Windows 7, Windows 8

Majento SiteAnalyzer is a web crawler (web spider) to scan the websites and check their technical and of SEO-parameters for errors and correct them effectively. Key features - Scanning of all pages of the site, as well as images, scripts and documents - Obtaining server response codes for...


Platforms: Windows, Windows 8, Windows 7, Windows Server

Cara-Pic v1.0 is an easy-to-use and completely customizable photo carousel with support for special effects, a variety of frame styles and includes a number of extra photo backdrops (backgrounds). Cara-Pic v1.0 provides an easy and convenient way to share your photo or image collection over the web.


Platforms: Windows

Pic-Share v1.0 is an easy-to-use and completely customizable photo album with support for special effects and up to 20 different photos or other images (more can be added if necessary or simply link individual albums together). Both the thumbnail previews and full-size views are completely...


Platforms: Windows

Shrink Pic resizes photos on-the-fly when sending with email, Instant Messenger or uploading to the web. It detects when you send large photo files, creates a duplicate, resizes and sends it instead. There are no operating instructions, just install, and it runs in the background. You can send...


Platforms: Windows

Schema Crawler project is a platform (database and OS) independent command-line tool to output your database schema and data in a readable form. The output is designed to be diff-ed with previous versions of your database schema. Schema Crawler is also an API that improves on standard JDBC...


Platforms: *nix