Hi,

While I was reading about the important Hadoop configuration properties, I
came across a state of doubt regarding the Java heap space properties for
the child tasks. According to my understanding, *mapred.child.java.opts* is
the overall heap size allocated to any task (map or reduce). Then when we
are setting *mapred.map.child.java.opts* and
*mapred.reduce.child.java.opts*separately, are they overriding the
*mapred.child.java.opts*?

For example, if I have the following configuration:
*mapred.child.java.opts = -Xmx1g

mapred.map.child.java.opts = -Xmx2g

mapred.reduce.child.java.opts = -Xmx512m
*Then how exactly the memory allocation is getting distributed between map
and reduce? My mapper gets more than the overall heap space as specified or
it is restricted to 1g?
Can some one help me understand this concept? Also, what are the other heap
space related properties which we can use with the above and how?

Thanks,
Gaurav

Hi Gaurav,

That's correct.  If the following was set:

*mapred.child.java.opts = -Xmx1g*
*mapred.map.child.java.opts = -Xmx2g*
*mapred.reduce.child.java.opts = -Xmx512m*

then:

1) -Xmx2G will be used for map tasks
2) -Xmx512m will be used for reduce tasks
3) the -Xmx1g will be ignored.

Kind Regards,

Anthony Rojas
On Fri, Mar 8, 2013 at 3:22 AM, Gaurav Dasgupta <[EMAIL PROTECTED]> wrote:

> Thanks for replying Harsh.
>
> So, it means that in my case of configuration, *mapred.child.java.opts > -Xmx1g* will be avoided completely and *mapred.map.child.java.opts > -Xmx2g* will be considered for map tasks and*mapred.reduce.child.java.opts = -Xmx512m
> * will be considered for reduce tasks. Right?
>
> Thanks,
> Gaurav
>