Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Question related to resource allocation in Yarn!


Copy link to this message
-
Question related to resource allocation in Yarn!
Hi,

I am trying to make a small poc on top of yarn.

Within the launched application master , I am trying to request for 50
containers and launch  a same task on those allocated containers.

My config : AM registration response minimumCapability {, memory: 1024,
virtual_cores: 1, }, maximumCapability {, memory: 8192, virtual_cores: 32,
},

1) I am asking for 1G mem and 1 core container to the RM. Ideally the RM
should return me 6 - 7 containers , but the response always returns with
only 2 containers.

Why is that ?

2) So , when in the first ask 2 containers are returned , then I again
required the RM for 50 - 2 = 48 containers. I keep getting 0 containers ,
even if the previously started containers have finished.

Why is that ?

Any link explaining the allocate request of RM would be very helpful.

Thanks,
Rahul