mrjob 0.3.0

mrjob is a Python package that helps you write and run Hadoop Streaming jobs.<br /><br />mrjob fully supports Amazon's Elastic MapReduce (EMR) service, which allows you to buy time on a Hadoop cluster on an hourly basis. It also works with your own Hadoop cluster.<br /><br />Some important features:<br /><br /> * Run jobs on EMR, your own Hadoop cluster, or locally (for testing).<br /> * Write multi-step jobs (one map-reduce step feeds into the next)<br /> * Duplicate your production environment inside Hadoop<br /> o Upload your source tree and put it in your job's $PYTHONPATH<br /> o Run make and other setup scripts<br /> o Set environment variables (e.g. $TZ)<br /> o Easily install python packages from tarballs (EMR only)<br /> o Setup handled transparently by mrjob.conf config file<br /> * Automatically interpret error logs from EMR<br /> * SSH tunnel to hadoop job tracker on EMR<br /> * Minimal setup<br /> o To run on EMR, set $AWS_ACCESS_KEY_ID and $AWS_SECRET_ACCESS_KEY<br /> o To run on your Hadoop cluster, install simplejson and make sure $HADOOP_HOME is set.<br /><br />Installation<br /><br />python setup.py install

Requirements:	No special requirements
Platforms:	*nix, Linux
Keyword:	Cluster,  Config,  Easily,  Environment,  Hadoop,  Handled,  Install,  Mrjobconf,  Package,  Packages,  Python,  Pythonpath,  Scripts,  Setup,  Tarballs,  Transparently,  Variables
Users rating:	0/10