Log::Parallel 0.303

Log::Parallel is the main driver module at the heart of a cluster computing framework used for batch log processing. It sets things up, figures out what jobs can run and in what order, and queues them up to run.<br /><br />Everything it does is driven from the configuration data, probably parsed by Config::YAMLMacros and validated by Log::Parallel::ConfigCheck.<br /><br />Only one program, process_logs, is expected to use this module. As such, documentation for the API for this module in particular is not particularly relevant, but this is as good a place as any to document the overall system.<br /><br />process_logs is the driver script for processing data logs through a series of jobs specified in a configuration file.<br /><br />SYNOPSIS<br /><br /> % bin/process_logs -c config_file<br /><br /> use Log::Parallel;<br /> use Log::Parallel::ConfigCheck qw(validate_config);<br /> use Proc::JobQueue::DependencyQueue;<br /><br /> opitons();<br /><br /> run($opts);<br /><br /> validate_config($config);<br /><br /> add_recnums($config);<br /><br /> $files_by_recnum{$_->{recnum}} = get_files_by_srec($_, $config->{hostsinfo})<br /> for @{$config->{sources}};<br /><br /> my $dependency_graph = make_dependency_graph(make_task_list($opts, $config, %files_by_recnum))<br /><br /> my $job_queue = new Proc::JobQueue::DependencyQueue(dependency_graph => $dependency_graph, hosts => [], hold_all => 1);<br /><br /> setup_slave_hosts($config, $job_queue);

Requirements:	No special requirements
Platforms:	*nix, Linux
Keyword:	Add R,  Binprocess Logs,  Config File,  Configuration,  Dependency Graph,  Driver,  Filesynopsis,  Job Queue,  Logparallel,  Logparallelconfigcheck,  Module,  Opitons,  Processing,  Procjobqueuedependencyqueue,  Qwvalidate Config,  Runopts,  Series,  Validate Configconfig
Users rating:	0/10