-Re: passing arguments to map reduce and queuing multiple jobs
Brock Noland 2012-02-29, 14:23
This question is for mapreduce-user not hbase-user.
On Wed, Feb 29, 2012 at 7:40 PM, T Vinod Gupta <[EMAIL PROTECTED]> wrote:
> whats the recommended way of passing arguments to m/r jobs? based on web
> examples, the mapper and reducer classes are static classes. so if you have
> to set some parameters that the individual instances of m/r runs need to
> access, whats the way? im trying to do it by creating a static initialize
> method in mapper and reducer and configuring static variables there. i am
> not sure if this is the best way.
Look at the configure method:
You are passed the job conf. You can set your parameters on the job conf.
> this brings me to next question - i need to create a always running daemon
> that schedules multiple m/r jobs everyday. it can just queue multiple jobs
> and wait for their completion. but then the above question becomes even
> more relevant. if static variables are used, then how can one queue
> multiple jobs on the same mapper/reducer?
I think you want to look at oozie: http://incubator.apache.org/oozie/
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/