Make Robots Txt

LinkChecker is a simple, very easy to use application specially designed to help you check websites and HTML documents for broken links. You just enter the URL or HTML doc to check, press Start and let LinkChecker take care of the rest. Features: recursive and multithreaded checking output in...

LinkChecker is a simple, very easy to use application specially designed to help you check websites and HTML documents for broken links. You just enter the URL or HTML doc to check, press Start and let LinkChecker take care of the rest. Features: recursive and multithreaded checking...

LinkChecker is a simple, very easy to use application specially designed to help you check websites and HTML documents for broken links. You just enter the URL or HTML doc to check, press Start and let LinkChecker take care of the rest. Features: recursive and multithreaded checking...


Platforms: Mac

Hello Engines! is a multi-awarded optimization tool which helps you to find all weaknesses and strengths of your website and to optimize it for search engines. It detects all errors in the HTML code, broken links, errors in the sitemap and robots.txt files etc. In addition, all important ranking...


Platforms: Windows

Sphider is a lightweight web spider and search engine written in PHP, using MySQL as its back end database. It is suitable for adding search functionality to small or medium sites (up to around 100,000 pages). It also works great as a tool for site analysis - finding broken links, gathering...


Platforms: Windows, Mac, *nix, PHP, BSD Solaris

Source code can be entered with for example ... or ....Supports highlighting for the following languages: ABAP, ActionScript, Ada, Apache Log, AppleScript, ASM, ASP, AutoIt, Bash, BlitzBasic, bnf, C, C (Mac), CAD DCL, CAD Lisp, CFDG, ColdFusion, C , C (QT), C#, CSS, D, Delphi, Diff, DIV, DOS,...


Platforms: Windows, Mac, *nix, PHP, BSD Solaris

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix (sometimes spelled heretrix, or misspelled or missaid as heratrix/heritix/ heretix/heratix) is an archaic word for heiress (woman who inherits). Since this crawler seeks to...


Platforms: Windows, Mac, *nix, Java, BSD Solaris

This is a simple and lightweight content management system for creating dynamic websites. Can be used for building blogs, wikis, ticket systems, documentation sites, or just plain old static webistes.Features of LitePublisher:- Image gallery- Audio player- Video player- Media RSS- RSS- Plugins-...


Platforms: PHP

Adds three more levels of privacy to the Options--Privacy page. 1. Blog visible to any logged in community member - "Network Users Only". 2. Blog visible only to registered users of blog - "Blog Members Only". 3. Blog visible only to administrators - "Admins Only".Mulitsite SuperAdmin can set an...


Platforms: PHP

The most complete all in one SEO solution for your WordPress blog, this plugin has a huge list of features, including: * Post title and meta description meta box to change these on a per post basis. * Taxonomy (tag, category & custom taxonomy) title and meta description support. * Google search...


Platforms: PHP

LinkChecker script allows you to check websites and HTML documents for broken links.Features: - recursive checking- multithreaded- output in colored or normal text, HTML, SQL, CSV, XML or a sitemap graph in different formats- HTTP/1.1, HTTPS, FTP, mailto:, news:, nntp:, Gopher, Telnet and local...


Platforms: Python

Current SEOpen features are all accessed through right-clicking an open area on a web page or by using the new toolbar; they include: * Yahoo Backlinks * Yahoo Domain-wide Backlinks * Pages in Yahoo Index * Google Backlinks * Google Cache * Pages in Google index * Google Translate to...


Platforms: Mac

A Python library for scraping things. Features include: * HTTP, HTTPS, FTP requests via an identical API * HTTP caching, compression and cookies * redirect following * request throttling * robots.txt compliance (optional) * robust error handling scrapelib is a project of Sunlight Labs (c)...


Platforms: *nix

Python programmatic web browser to fetch data and test web sites. Bug reporting and features asking are welcome, use the the bug tracker (Issues). Pynav on pypi : http://pypi.python.org/pypi/pynav/ Features: * Post authentication * User agent support * Automatic cookie handling * HTTP Basic...


Platforms: *nix

SEO & Website Analysis is a Firefox extension that provides a very deep SEO & Website Analysis report for any given website. When you click on the SEO WooRank icon, we will generate the SEO report for your website. The Ad-on shows a "W" icon on the Firefox status bar (bottom right), and in...


Platforms: *nix

mechanize offers stateful programmatic web browsing, build after Andy Lester's Perl module WWW::Mechanize. The library is layered: mechanize.Browser (stateful web browser), mechanize.UserAgent (configurable URL opener), plus urllib2 handlers. Features include: ftp:, http: and file: URL schemes,...


Platforms: *nix

SEOpen is an extension I made for Firefox to help with search engine optimization. It provides many helpful tools at the click of a mouse. SEOpen features are accessed through right-clicking an open area on a web page or by using the included toolbar; they include: * Yahoo Backlinks * Yahoo...


Platforms: *nix

pyramid_assetviews is a small module that allows you to add Root-Relative views to your project. This enables you to serve assets that need to be accessed from the root of your domain such as robots.txt or favicon.ico without having to setup a static media server such as nginx or lighttpd.


Platforms: *nix

Is link building taking too much of your time? Try LinkAssistant - a multi-functional app that lets you do a variety of link building tasks. Use this ultimate SEO tool to: - Find thousands of link opportunities using over 10 different methods of prospect search; - Collect the potential...


Platforms: Windows, Windows 8, Windows 7, Windows Server

Sitemap Writer Pro is an easy- to -use program, fast and efficient, which offers 6 different types of sitemaps (Standard Sitemap, Google News, Google Video, Google Code Search, Google Geo, Google Mobile) to properly index every website, generate an xml sitemap and upload it to all search engines...


Platforms: Windows, Windows 7