Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: Allocating Containers on a particular Node in Yarn


Copy link to this message
-
Re: Allocating Containers on a particular Node in Yarn
Hi Sandy,

No it is not working for me.  As mentioned earlier, AMRMClient is not
respecting the locality for node and rack in *Line 361 and 374 *and is
set to true always.
I am requesting for one container with specified node and rack and
relaxed locality false.

Thanks
-Gaurav

On 11/13/2013 4:24 PM, Sandy Ryza wrote:
> [moving to user list]
>
> Right.  relaxLocality needs to be set on the next level up.  It
> determines whether locality can be relaxed to that level.  Confusing,
> I know.  If you are using AMRMClient, you should be able to accomplish
> what you're looking for by creating an AMRMClient.ContainerRequest
> that specifies a node and sets relax locality to false.  Is that not
> working for you?
>
> -Sandy
>
>
> On Wed, Nov 13, 2013 at 4:19 PM, gaurav <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
>
>     Hi Sandy,
>
>     If I set relaxlocality to true with host name, I don't get the
>     container on specified host even though node has the resources.
>     I am using AMRMClient, only thing is I made following changes to
>     get the containers on the specified node.
>     /
>     //Line: 361 //
>     //Original//
>     // addResourceRequest(req.getPriority(), node,
>     req.getCapability(), req, true);//
>     //Modifiled//
>     // addResourceRequest(req.getPriority(), node,
>     req.getCapability(), req, req.getRelaxLocality());//
>     //
>     //Line 374//
>     //Original//
>     // addResourceRequest(req.getPriority(), rack,
>     req.getCapability(), req, true);//
>     //Modifiled//
>     // addResourceRequest(req.getPriority(), rack,
>     req.getCapability(), req, req.getRelaxLocality());//
>     ///
>
>
>     Thanks
>     -Gaurav
>
>
>     On 11/13/2013 4:02 PM, Sandy Ryza wrote:
>
>         Yeah, specifying a host name with relaxLocality is meaningful.
>          Schedulers
>         use delay scheduling (
>         http://www.cs.berkeley.edu/~matei/talks/2010/eurosys_delaysched.pdf
>         <http://www.cs.berkeley.edu/%7Ematei/talks/2010/eurosys_delaysched.pdf>)
>         to
>         achieve locality when relaxLocality is on.  But it is turned
>         off by
>         default.  The individual scheduler docs have specifics on how
>         to configure
>         it.
>
>         Guarav,
>         Using ResourceRequests directly is not straightforward and
>         error prone.  Is
>         there a reason that AMRMClient is unsuitable for your needs?
>
>         -Sandy
>
>
>
>         On Wed, Nov 13, 2013 at 3:55 PM, Thomas Weise
>         <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>wrote:
>
>             Is it possible to specify a particular node and have RM
>             fallback to an
>             different node only after making an attempt to allocate
>             for the requested
>             node? In other words, is the combination of specific host
>             name and
>             relaxLocality=TRUE meaningful at all?
>
>             Thanks.
>
>
>             On Wed, Nov 13, 2013 at 3:23 PM, Alejandro Abdelnur
>             <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
>
>                 wrote:
>                 Gaurav,
>
>                 Setting relaxLocality to FALSE should do it.
>
>                 thanks.
>
>
>                 On Wed, Nov 13, 2013 at 2:58 PM, gaurav
>                 <[EMAIL PROTECTED]
>                 <mailto:[EMAIL PROTECTED]>> wrote:
>
>                       Hi,
>                       I am trying to allocate containers on a
>                     particular node in Yarn but
>
>             Yarn
>
>                     is returning me containers on different node
>                     although the requested
>
>             node
>
>                     has resources available.
>
>                     I checked into the allocate(AllocateRequest
>                     request) function of
>                     ApplicationMasterService and my request is as follows
>
>                     *request: ask { priority { priority: 1 }