Robots Txt Pingserver Php
On WP-MU installations it allows for every blog to have its own robots.txt file.InstallationUnpack and upload it to the /wp-content/plugins/ directory.Activate the plugin through the 'Plugins' menu in WordPress. Requirements: - WordPress 2.0 up to 2.5
Platforms: Windows, Mac, *nix, PHP, BSD Solaris
License: Freeware | Download (55): KB Robots.txt Download |
A robots.txt file lets search engines (Google, Yahoo, MSN, etc) know which pages on your site you don't want them to index. This is helpful in keeping them from going to pages that will confuse them and/or won't provide any added benefit to you from having indexed.Since nobody has put one in here...
Platforms: Windows, *nix, PHP, BSD
License: Freeware | Download (50): robots.txt Sample File Download |
This module generates the robots.txt file dynamically and gives you the chance to edit it, on a per-site basis, from the web UI.Installation - Unpack to your Drupal site modules directory (usually /sites/all/modules/) - Enable from Administer > Site Building > ModulesĀ Requirements: - Drupal...
Platforms: Windows, Mac, *nix, PHP, BSD Solaris
License: Freeware | Download (54): RobotsTxt Download |
Visual SEO Studio is a desktop SEO Audit Tool, a Visual approach to make SEO tasks quick and easy.
The program lets you Crawl one or more sites in parallel with its Interactive Web Spider, Spot on-site issues at a glance, Save Page Screenshot miniatures and see where the fold lays, visually...
Platforms: Windows, Windows 7, Windows Server
License: Freeware | Size: 9.83 MB | Download (447): Visual SEO Studio Download |
TNT Forum is a Free and Open Source TXT-Based PHP forum, with templates and multi-language support, and many others features.
Platforms: Windows, Mac, *nix, PHP, BSD Solaris
License: Freeware | Download (62): TNT Forum Download |
A free PHP script, converts RSS feeds to HTML. The rss2html.php script allows webmasters to display RSS feeds on their website. RSS or Really Simple Syndication, as it is commonly known, is a technology that gives webmasters the ability to to easily distribute and publish syndicated content on...
Platforms: Windows, Mac, *nix
License: Freeware | Size: 28.13 KB | Download (286): rss2html Download |
Sphider is a lightweight web spider and search engine written in PHP, using MySQL as its back end database. It is suitable for adding search functionality to small or medium sites (up to around 100,000 pages). It also works great as a tool for site analysis - finding broken links, gathering...
Platforms: Windows, Mac, *nix, PHP, BSD Solaris
License: Freeware | Download (60): Sphider Download |
Installation - Unpack in your modules folder (usually /sites/all/modules/) and enable under Administer > Site Building > Modules. Requirements: - Drupal 6.x
Platforms: Windows, Mac, *nix, PHP, BSD Solaris
License: Freeware | Download (48): Robots parser Download |
RobotPack is a free robot-exclusion management tool to inform robots and other automated search engine tools not to index parts of your Web site. RobotPack provides an FTP access to your Web site so you can easily select files and directories which you don't want indexed by search engines....
Platforms: Windows
License: Freeware | Size: 5.72 MB | Download (97): RobotPack Download |
Source code can be entered with for example ... or ....Supports highlighting for the following languages: ABAP, ActionScript, Ada, Apache Log, AppleScript, ASM, ASP, AutoIt, Bash, BlitzBasic, bnf, C, C (Mac), CAD DCL, CAD Lisp, CFDG, ColdFusion, C , C (QT), C#, CSS, D, Delphi, Diff, DIV, DOS,...
Platforms: Windows, Mac, *nix, PHP, BSD Solaris
License: Freeware | Download (54): GeSHi Filter for syntax highlighting Download |
This simple robots.txt file will help deter some of the unwanted bots that spider your site for email adresses and things of that nature that result in spam. This works for all websites and it is not just limited to phpBB.This has also been configured to allow google adsense bots to spider your...
Platforms: Windows, *nix, PHP, BSD
License: Freeware | Download (54): Block Bad Bots Download |
Adds three more levels of privacy to the Options--Privacy page. 1. Blog visible to any logged in community member - "Network Users Only". 2. Blog visible only to registered users of blog - "Blog Members Only". 3. Blog visible only to administrators - "Admins Only".Mulitsite SuperAdmin can set an...
Platforms: PHP
License: Freeware | Size: 10 KB | Download (54): More Privacy Options Download |
TracRobotsTxt is a module that will serve a robots.txt file from Trac. Mostly useful to tracd users, but works on anything. Just put the data you want in the wiki page RobotsTxt. Configuration: To enable the plugin: [components] robotstxt.* = enabled A typical RobotsTxt wiki page will look...
Platforms: *nix
License: Freeware | Size: 10.24 KB | Download (39): TracRobotsTxt Download |
The Revisionist is a tool for extracting and indexing hidden metadata (such as deleted or modified text) from large collections of MS Word files. It can operate whole Web sites or SMB or NFS directories. The Revisionist project is handy for pen-testing, or it can be used just to spot...
Platforms: *nix
License: Freeware | Size: 16.38 KB | Download (97): The Revisionist Download |
Crawl the web with this free ActiveX web bot component. Advanced features include caching, "avoid" patterns, and robots.txt compliance.
Platforms: Windows
License: Freeware | Size: 532.48 KB | Download (510): Web Bot Programming Library Download |
Crawl a website with this free ActiveX spidering component. Advanced features include caching, "avoid" patterns, and robots.txt compliance.
Platforms: Windows
License: Freeware | Size: 532.48 KB | Download (461): Chilkat Spider ActiveX Download |
LinkChecker is a simple, very easy to use application specially designed to help you check websites and HTML documents for broken links. You just enter the URL or HTML doc to check, press Start and let LinkChecker take care of the rest. Features: recursive and multithreaded checking output in...
Platforms: Windows
License: Freeware | Size: 10.5 MB | Download (48): LinkChecker Download |
LinkChecker is a simple, very easy to use application specially designed to help you check websites and HTML documents for broken links. You just enter the URL or HTML doc to check, press Start and let LinkChecker take care of the rest. Features: recursive and multithreaded checking output in...
Platforms: Windows
License: Freeware | Size: 10.5 MB | Download (46): LinkChecker Portable Download |
LinkChecker is a simple, very easy to use application specially designed to help you check websites and HTML documents for broken links.
You just enter the URL or HTML doc to check, press Start and let LinkChecker take care of the rest.
Features:
recursive and multithreaded checking...
Platforms: Linux
License: Freeware | Size: 10.5 MB | Download (48): LinkChecker for Linux Download |
LinkChecker is a simple, very easy to use application specially designed to help you check websites and HTML documents for broken links.
You just enter the URL or HTML doc to check, press Start and let LinkChecker take care of the rest.
Features:
recursive and multithreaded checking...
Platforms: Mac
License: Freeware | Size: 10.5 MB | Download (48): LinkChecker for Mac OS X Download |