Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Task status query


Copy link to this message
-
Re: Task status query
Harsh J 2013-09-20, 03:20
Hi John,

YARN tasks can be more than simple executables. In case of MR, for
example, tasks talk to the AM and report their individual progress and
counters back to it, via a specific protocol (over the network),
giving the AM more data to compute an near-accurate global progress.

On Fri, Sep 20, 2013 at 12:18 AM, John Lilley <[EMAIL PROTECTED]> wrote:
> How does a YARN application master typically query ongoing status (like
> percentage completion) of its tasks?
>
> I would like to be able to ultimately relay information to the user like:
>
> 100 tasks are scheduled
>
> 10 tasks are complete
>
> 4 tasks are running and they are (4%, 10%, 50%, 70%) complete
>
> But, given that YARN tasks are simply executables, how can the AM even get
> at this information?  Can the AM get access to stdout/stderr?
>
> Thanks
>
> John
>
>

--
Harsh J