Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Question related to resource allocation in Yarn!


Copy link to this message
-
RE: Question related to resource allocation in Yarn!
Hi Rahul,

    Could you tell me, what is the version you are using?

·         If you want a container, you need to issue 3 resource requests (1-node local, 1-rack local and 1-Any(*) ). If you are using 2.1.0-beta<https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=ASC&sorter/field=priority&pid=12313722&customfield_12310320=12324029> or later versions, you can set the Relax Locality flag to false for getting only on the specified host.
Can you also share the code how you are requesting for containers…so that we can help you better..

Thanks
Devaraj k

From: Rahul Bhattacharjee [mailto:[EMAIL PROTECTED]]
Sent: 06 September 2013 09:43
To: [EMAIL PROTECTED]
Subject: Re: Question related to resource allocation in Yarn!

I could progress a bit on this.

I was not setting responseId while asking for containers.
Still I have one question as why I am only been allocated two containers whereas node manager can run more containers.

Response while registering the application master -
AM registration response minimumCapability {, memory: 1024, virtual_cores: 1, }, maximumCapability {, memory: 8192, virtual_cores: 32, },
Thanks,
Rahul

On Thu, Sep 5, 2013 at 8:33 PM, Rahul Bhattacharjee <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Hi,
I am trying to make a small poc on top of yarn.
Within the launched application master , I am trying to request for 50 containers and launch  a same task on those allocated containers.
My config : AM registration response minimumCapability {, memory: 1024, virtual_cores: 1, }, maximumCapability {, memory: 8192, virtual_cores: 32, },
1) I am asking for 1G mem and 1 core container to the RM. Ideally the RM should return me 6 - 7 containers , but the response always returns with only 2 containers.
Why is that ?
2) So , when in the first ask 2 containers are returned , then I again required the RM for 50 - 2 = 48 containers. I keep getting 0 containers , even if the previously started containers have finished.
Why is that ?
Any link explaining the allocate request of RM would be very helpful.

Thanks,
Rahul