Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> whitelist feature of YARN


+
Krishna Kishore Bonagiri 2013-07-08, 13:48
+
Sandy Ryza 2013-07-08, 17:23
+
Krishna Kishore Bonagiri 2013-07-09, 09:54
+
Sandy Ryza 2013-08-07, 06:07
Copy link to this message
-
Re: whitelist feature of YARN
Hi Sandy,

  Thanks for the reply and it is good to know YARN-521 is done! Please
answer my following questions

1) when is 2.1.0-beta going to be released? is it soon or do you suggest me
take it from the trunk or is there a recent release candidate available?

2) I have recently changed my application to use the new Asynchronous
interfaces. I am hoping it works with that too, correct me if I am wrong.

3) Change in interface:

The old interface for ContainerRequest constructor used to be this:

 public ContainerRequest(Resource capability, String[] nodes,
        String[] racks, Priority priority, int containerCount);

where as now it is changed to

a) public ContainerRequest(Resource capability, String[] nodes,
        String[] racks, Priority priority)
&

b) public ContainerRequest(Resource capability, String[] nodes,
        String[] racks, Priority priority, boolean relaxLocality)

that means the old argument containerCount is gone! How would I be able to
specify how many containers do I need?

-Kishore
On Wed, Aug 7, 2013 at 11:37 AM, Sandy Ryza <[EMAIL PROTECTED]> wrote:

> YARN-521, which brings whitelisting to the AMRMClient APIs, is now
> included in 2.1.0-beta.  Check out the doc for the relaxLocality paramater
> in ContainerRequest in AMRMClient:
> https://github.com/apache/hadoop-common/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/AMRMClient.java and
> I can help clarify here if anything's confusing.
>
> -Sandy
>
>
> On Tue, Jul 9, 2013 at 2:54 AM, Krishna Kishore Bonagiri <
> [EMAIL PROTECTED]> wrote:
>
>> Hi Sandy,
>>
>>   Yes, I have been using AMRMClient APIs. I am planning to shift to
>> whatever way is this white list feature is supported with. But am not sure
>> what is meant by submitting ResourceRequests directly to RM. Can you please
>> elaborate on this or give me a pointer to some example code on how to do
>> it...
>>
>>    Thanks for the reply,
>>
>> -Kishore
>>
>>
>> On Mon, Jul 8, 2013 at 10:53 PM, Sandy Ryza <[EMAIL PROTECTED]>wrote:
>>
>>> Hi Krishna,
>>>
>>> From your previous email, it looks like you are using the AMRMClient
>>> APIs.  Support for whitelisting is not yet supported through them.  I am
>>> working on this in YARN-521, which should be included in the next release
>>> after 2.1.0-beta.  If you are submitting ResourceRequests directly to the
>>> RM, you can whitelist a node by
>>> * setting the relaxLocality flag on the node-level ResourceRequest to
>>> true
>>> * setting the relaxLocality flag on the corresponding rack-level
>>> ResourceRequest to false
>>> * setting the relaxLocality flag on the corresponding any-level
>>> ResourceRequest to false
>>>
>>> -Sandy
>>>
>>>
>>> On Mon, Jul 8, 2013 at 6:48 AM, Krishna Kishore Bonagiri <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> Hi,
>>>>
>>>>   Can someone please point to some example code of how to use the
>>>> whitelist feature of YARN, I have recently got RC1 for hadoop-2.1.0-beta
>>>> and want to use this feature.
>>>>
>>>>   It would be great if you can point me to some description of what
>>>> this white listing feature is, I have gone through some JIRA logs related
>>>> to this but more concrete explanation would be helpful.
>>>>
>>>> Thanks,
>>>> Kishore
>>>>
>>>
>>>
>>
>
+
Sandy Ryza 2013-08-07, 07:05
+
Krishna Kishore Bonagiri 2013-08-07, 09:06
+
Arun C Murthy 2013-08-07, 16:53
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB