Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> whitelist feature of YARN


+
Krishna Kishore Bonagiri 2013-07-08, 13:48
+
Sandy Ryza 2013-07-08, 17:23
+
Krishna Kishore Bonagiri 2013-07-09, 09:54
+
Sandy Ryza 2013-08-07, 06:07
Copy link to this message
-
Re: whitelist feature of YARN
Hi Sandy,

  Thanks for the reply and it is good to know YARN-521 is done! Please
answer my following questions

1) when is 2.1.0-beta going to be released? is it soon or do you suggest me
take it from the trunk or is there a recent release candidate available?

2) I have recently changed my application to use the new Asynchronous
interfaces. I am hoping it works with that too, correct me if I am wrong.

3) Change in interface:

The old interface for ContainerRequest constructor used to be this:

 public ContainerRequest(Resource capability, String[] nodes,
        String[] racks, Priority priority, int containerCount);

where as now it is changed to

a) public ContainerRequest(Resource capability, String[] nodes,
        String[] racks, Priority priority)
&

b) public ContainerRequest(Resource capability, String[] nodes,
        String[] racks, Priority priority, boolean relaxLocality)

that means the old argument containerCount is gone! How would I be able to
specify how many containers do I need?

-Kishore
On Wed, Aug 7, 2013 at 11:37 AM, Sandy Ryza <[EMAIL PROTECTED]> wrote:

> YARN-521, which brings whitelisting to the AMRMClient APIs, is now
> included in 2.1.0-beta.  Check out the doc for the relaxLocality paramater
> in ContainerRequest in AMRMClient:
> https://github.com/apache/hadoop-common/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/AMRMClient.java and
> I can help clarify here if anything's confusing.
>
> -Sandy
>
>
> On Tue, Jul 9, 2013 at 2:54 AM, Krishna Kishore Bonagiri <
> [EMAIL PROTECTED]> wrote:
>
>> Hi Sandy,
>>
>>   Yes, I have been using AMRMClient APIs. I am planning to shift to
>> whatever way is this white list feature is supported with. But am not sure
>> what is meant by submitting ResourceRequests directly to RM. Can you please
>> elaborate on this or give me a pointer to some example code on how to do
>> it...
>>
>>    Thanks for the reply,
>>
>> -Kishore
>>
>>
>> On Mon, Jul 8, 2013 at 10:53 PM, Sandy Ryza <[EMAIL PROTECTED]>wrote:
>>
>>> Hi Krishna,
>>>
>>> From your previous email, it looks like you are using the AMRMClient
>>> APIs.  Support for whitelisting is not yet supported through them.  I am
>>> working on this in YARN-521, which should be included in the next release
>>> after 2.1.0-beta.  If you are submitting ResourceRequests directly to the
>>> RM, you can whitelist a node by
>>> * setting the relaxLocality flag on the node-level ResourceRequest to
>>> true
>>> * setting the relaxLocality flag on the corresponding rack-level
>>> ResourceRequest to false
>>> * setting the relaxLocality flag on the corresponding any-level
>>> ResourceRequest to false
>>>
>>> -Sandy
>>>
>>>
>>> On Mon, Jul 8, 2013 at 6:48 AM, Krishna Kishore Bonagiri <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> Hi,
>>>>
>>>>   Can someone please point to some example code of how to use the
>>>> whitelist feature of YARN, I have recently got RC1 for hadoop-2.1.0-beta
>>>> and want to use this feature.
>>>>
>>>>   It would be great if you can point me to some description of what
>>>> this white listing feature is, I have gone through some JIRA logs related
>>>> to this but more concrete explanation would be helpful.
>>>>
>>>> Thanks,
>>>> Kishore
>>>>
>>>
>>>
>>
>
+
Sandy Ryza 2013-08-07, 07:05
+
Krishna Kishore Bonagiri 2013-08-07, 09:06
+
Arun C Murthy 2013-08-07, 16:53