Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Re: Semantics of ApplicationResourceUsageReport


Copy link to this message
-
Re: Semantics of ApplicationResourceUsageReport
Sandy Ryza 2013-09-21, 08:22
Hi Albert,

You're correct about used.

Reserved is a little bit more arcane - it refers to a mechanism that
schedulers use to prevent applications with larger container sizes from
starving.  Applications place container "reservations" on nodes, and no
other containers can be placed on the node until the reservation is
fulfilled.

Needed is the amount requested that is not yet used.

-Sandy

On Sat, Sep 14, 2013 at 3:18 AM, Albert Shau <[EMAIL PROTECTED]> wrote:

> Hi,
>
> Given an application id the YarnClient can return an
> ApplicationResourceUsageReport that gives information about resources that
> are 'used', 'reserved', and 'needed'.  I'm trying to get a better
> understanding of what determines whether a resource is 'used', 'reserved',
> or 'needed'.  If I request a container with some resource, is that 'used'
> once the resource manager has given the container to the application master
> even if the application master has not started the container yet?  In what
> situation is a resource 'reserved'?  Is 'needed' the sum of the two, or
> does it have some other meaning?
>
> Thanks,
> Albert
>