-Re: how to start the customized Child JVM task process of m/r?
Owen O'Malley 2011-04-26, 15:05
On Tue, Apr 26, 2011 at 6:02 AM, Juwei Shi <[EMAIL PROTECTED]> wrote:
As you know, Hadoop MapReduce start child JVM processes to run task. I want
> to start the process of m/r task myself thus I can pass some OS level
> parameters to the JVM process.
That currently isn't possible. The best that is available
is mapred.child.java.opts, which adds options to the child jvm.
Alternatively, you could fork a subprocess out of the map or reduce like
streaming does. Actually, I guess you could use streaming itself and get
control over how your code runs.
In MapReduce v2, which is described here
the user gets a lot more control over the task launching. In particular,
mapreduce becomes a user-land library controlled by the user.