Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: YARN - specify hosts in ContainerRequest


Copy link to this message
-
Re: YARN - specify hosts in ContainerRequest
Brian C. Huffman 2013-04-12, 19:08
I get a container, but not on the node I'm asking for.

Thanks,
Brian

On 04/12/2013 03:01 PM, Sandy Ryza wrote:
> What do you mean when you say it doesn't seem to use the code?  That
> you're not getting containers back?
>
> -Sandy
>
> On Fri, Apr 12, 2013 at 7:44 AM, Brian C. Huffman
> <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>
> wrote:
>
>     Yes, this is exactly the DistributedShell example in the 2.0.3
>     source (except for the added specification of containerNode).
>
>     So, it's doing the following:
>               ContainerRequest containerAsk >     setupContainerAskForRM(askCount);
>               resourceManager.addContainerRequest(containerAsk);
>
>
>             // Send the request to RM
>             LOG.info("Asking RM for containers" + ", askCount=" +
>     askCount);
>             AMResponse amResp = sendContainerAskToRM();
>
>     FYI - the sendContainerAskToRM calls the allocate(progress).
>
>     I don't see any NPEs in the resourcemanager's logs.
>
>     Regards,
>     Brian
>
>
>     On 04/11/2013 02:50 PM, Harsh J wrote:
>
>         Hi Brian,
>
>         Your way of creation seems fine. Are you "adding" this to the
>         AMRMClient somewhere though, via [1], right before you call
>         allocate(progress)? Cause otherwise this object's not used at all.
>
>         P.s. I'm not sure we'll handle that null for racks all that
>         well in
>         certain places yet; I've noticed errors in my app when I've missed
>         passing elements or passed them as nulls, but I didn't rebase
>         it onto
>         2.0.3 yet so haven't reported anything so far - will do so
>         soon. I'd
>         check the RM log to see if there's any WARN logged there with
>         an NPE.
>
>         [1] -
>         http://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/AMRMClient.html#addContainerRequest(org.apache.hadoop.yarn.client.AMRMClient.ContainerRequest)
>         <http://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/AMRMClient.html#addContainerRequest%28org.apache.hadoop.yarn.client.AMRMClient.ContainerRequest%29>
>
>         On Thu, Apr 11, 2013 at 11:30 PM, Brian C. Huffman
>         <[EMAIL PROTECTED]
>         <mailto:[EMAIL PROTECTED]>> wrote:
>
>             All,
>
>             I'm starting to work with YARN and I'm trying to make some
>             changes to the
>             distributedshell example.
>
>             Specifically I'm trying to set the host where the
>             distributed shell
>             container will run.  I added the following code into
>             ApplicationMaster.java:
>                  final String[] containerNodes = { "node1" };
>
>                  LOG.info("Trying to request node: " + containerNodes[0]);
>                  ContainerRequest request = new
>             ContainerRequest(capability,
>             containerNodes, null,
>                      pri, numContainers);
>
>             And while the log output shows this code is being executed
>             ("Trying to
>             request..."), it doesn't seem to use it.  Is this
>             functionality that isn't
>             complete yet in the 2.0.3 yarn alpha?  Or am I doing
>             something wrong?
>
>             Thanks,
>             Brian
>
>
>
>
>
>