Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Task status query

Copy link to this message
Re: Task status query
Harsh J 2013-09-20, 03:20
Hi John,

YARN tasks can be more than simple executables. In case of MR, for
example, tasks talk to the AM and report their individual progress and
counters back to it, via a specific protocol (over the network),
giving the AM more data to compute an near-accurate global progress.

On Fri, Sep 20, 2013 at 12:18 AM, John Lilley <[EMAIL PROTECTED]> wrote:
> How does a YARN application master typically query ongoing status (like
> percentage completion) of its tasks?
> I would like to be able to ultimately relay information to the user like:
> 100 tasks are scheduled
> 10 tasks are complete
> 4 tasks are running and they are (4%, 10%, 50%, 70%) complete
> But, given that YARN tasks are simply executables, how can the AM even get
> at this information?  Can the AM get access to stdout/stderr?
> Thanks
> John

Harsh J