Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: YARN - specify hosts in ContainerRequest


Copy link to this message
-
Re: YARN - specify hosts in ContainerRequest
I get a container, but not on the node I'm asking for.

Thanks,
Brian

On 04/12/2013 03:01 PM, Sandy Ryza wrote:
> What do you mean when you say it doesn't seem to use the code?  That
> you're not getting containers back?
>
> -Sandy
>
> On Fri, Apr 12, 2013 at 7:44 AM, Brian C. Huffman
> <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>
> wrote:
>
>     Yes, this is exactly the DistributedShell example in the 2.0.3
>     source (except for the added specification of containerNode).
>
>     So, it's doing the following:
>               ContainerRequest containerAsk >     setupContainerAskForRM(askCount);
>               resourceManager.addContainerRequest(containerAsk);
>
>
>             // Send the request to RM
>             LOG.info("Asking RM for containers" + ", askCount=" +
>     askCount);
>             AMResponse amResp = sendContainerAskToRM();
>
>     FYI - the sendContainerAskToRM calls the allocate(progress).
>
>     I don't see any NPEs in the resourcemanager's logs.
>
>     Regards,
>     Brian
>
>
>     On 04/11/2013 02:50 PM, Harsh J wrote:
>
>         Hi Brian,
>
>         Your way of creation seems fine. Are you "adding" this to the
>         AMRMClient somewhere though, via [1], right before you call
>         allocate(progress)? Cause otherwise this object's not used at all.
>
>         P.s. I'm not sure we'll handle that null for racks all that
>         well in
>         certain places yet; I've noticed errors in my app when I've missed
>         passing elements or passed them as nulls, but I didn't rebase
>         it onto
>         2.0.3 yet so haven't reported anything so far - will do so
>         soon. I'd
>         check the RM log to see if there's any WARN logged there with
>         an NPE.
>
>         [1] -
>         http://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/AMRMClient.html#addContainerRequest(org.apache.hadoop.yarn.client.AMRMClient.ContainerRequest)
>         <http://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/AMRMClient.html#addContainerRequest%28org.apache.hadoop.yarn.client.AMRMClient.ContainerRequest%29>
>
>         On Thu, Apr 11, 2013 at 11:30 PM, Brian C. Huffman
>         <[EMAIL PROTECTED]
>         <mailto:[EMAIL PROTECTED]>> wrote:
>
>             All,
>
>             I'm starting to work with YARN and I'm trying to make some
>             changes to the
>             distributedshell example.
>
>             Specifically I'm trying to set the host where the
>             distributed shell
>             container will run.  I added the following code into
>             ApplicationMaster.java:
>                  final String[] containerNodes = { "node1" };
>
>                  LOG.info("Trying to request node: " + containerNodes[0]);
>                  ContainerRequest request = new
>             ContainerRequest(capability,
>             containerNodes, null,
>                      pri, numContainers);
>
>             And while the log output shows this code is being executed
>             ("Trying to
>             request..."), it doesn't seem to use it.  Is this
>             functionality that isn't
>             complete yet in the 2.0.3 yarn alpha?  Or am I doing
>             something wrong?
>
>             Thanks,
>             Brian
>
>
>
>
>
>

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB