Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> YARN - specify hosts in ContainerRequest


+
Brian C. Huffman 2013-04-11, 18:00
+
Brian C. Huffman 2013-04-12, 14:44
Copy link to this message
-
Re: YARN - specify hosts in ContainerRequest
What do you mean when you say it doesn't seem to use the code?  That you're
not getting containers back?

-Sandy

On Fri, Apr 12, 2013 at 7:44 AM, Brian C. Huffman <
[EMAIL PROTECTED]> wrote:

> Yes, this is exactly the DistributedShell example in the 2.0.3 source
> (except for the added specification of containerNode).
>
> So, it's doing the following:
>           ContainerRequest containerAsk = setupContainerAskForRM(**
> askCount);
>           resourceManager.**addContainerRequest(**containerAsk);
>
>
>         // Send the request to RM
>         LOG.info("Asking RM for containers" + ", askCount=" + askCount);
>         AMResponse amResp = sendContainerAskToRM();
>
> FYI - the sendContainerAskToRM calls the allocate(progress).
>
> I don't see any NPEs in the resourcemanager's logs.
>
> Regards,
> Brian
>
>
> On 04/11/2013 02:50 PM, Harsh J wrote:
>
>> Hi Brian,
>>
>> Your way of creation seems fine. Are you "adding" this to the
>> AMRMClient somewhere though, via [1], right before you call
>> allocate(progress)? Cause otherwise this object's not used at all.
>>
>> P.s. I'm not sure we'll handle that null for racks all that well in
>> certain places yet; I've noticed errors in my app when I've missed
>> passing elements or passed them as nulls, but I didn't rebase it onto
>> 2.0.3 yet so haven't reported anything so far - will do so soon. I'd
>> check the RM log to see if there's any WARN logged there with an NPE.
>>
>> [1] - http://hadoop.apache.org/docs/**current/api/org/apache/hadoop/**
>> yarn/client/AMRMClient.html#**addContainerRequest(org.**
>> apache.hadoop.yarn.client.**AMRMClient.ContainerRequest)<http://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/AMRMClient.html#addContainerRequest(org.apache.hadoop.yarn.client.AMRMClient.ContainerRequest)>
>>
>> On Thu, Apr 11, 2013 at 11:30 PM, Brian C. Huffman
>> <[EMAIL PROTECTED]> wrote:
>>
>>> All,
>>>
>>> I'm starting to work with YARN and I'm trying to make some changes to the
>>> distributedshell example.
>>>
>>> Specifically I'm trying to set the host where the distributed shell
>>> container will run.  I added the following code into
>>> ApplicationMaster.java:
>>>      final String[] containerNodes = { "node1" };
>>>
>>>      LOG.info("Trying to request node: " + containerNodes[0]);
>>>      ContainerRequest request = new ContainerRequest(capability,
>>> containerNodes, null,
>>>          pri, numContainers);
>>>
>>> And while the log output shows this code is being executed ("Trying to
>>> request..."), it doesn't seem to use it.  Is this functionality that
>>> isn't
>>> complete yet in the 2.0.3 yarn alpha?  Or am I doing something wrong?
>>>
>>> Thanks,
>>> Brian
>>>
>>>
>>
>>
>
>
+
Anand Mundada 2014-02-14, 18:14