-Re: How mapred.map.child.java.opts influence the Container resource allocation
YouPeng Yang 2013-02-05, 00:44
Thanks a lot.
With a container size of 2G, the max heap size i can allocate is ~1.6G.
is the heap exclusively possessed by the container ,or shared by all
which belong to a AM.
I also notice that there are several containers allocated in a NM.
however only one or a few of them would process the job,
(State:NEW --> LOCALIZING --> LOCALIZED --> RUNNING --> KILLING -->
Others were killed (State:NEW --> LOCALIZING --> LOCALIZED --> RUNNING
--> KILLING --> CONTAINER_CLEANEDUP_AFTER_KILL--> DONE).
So i wonder what is the container allocation policy,and how many
containers would be allocated when a job is submitted.
2013/2/4 Arun C Murthy <[EMAIL PROTECTED]>
> The *container* is a generic concept which allows the ResourceManager to
> provide you a specific amount of resources (memory, cpu etc.).
> mapred.map.child.java.opts is part of the cmd-line of the Map task's jvm
> which specifies the heap size.
> So, in YARN, mapred.map.child.java.opts should be less than memory
> allocated to container.
> Remember that container memory is physical memory of the unix process,
> while mapred.map.child.java.opts is heap size which doesn't account for
> non-heap memory.
> So, with a container size of 2G, the max heap size you can allocate is
> ~1.6G or so.
> On Feb 4, 2013, at 3:38 AM, YouPeng Yang wrote:
> Hi All
> I use YARN and MR.I am confused by the relationship between
> mapred.map.child.java.opts and Container resource allocation.
> if I setup mapred.map.child.java.opts = -Xmx2048m ,on nodemanager
> how Containers will be allocated according to this settings.
> what about mapred.reduce.child.java.opts?
> Youpeng Yang
> Arun C. Murthy
> Hortonworks Inc.