Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Task status query


+
John Lilley 2013-09-19, 18:48
Copy link to this message
-
Re: Task status query
Hi John,

YARN tasks can be more than simple executables. In case of MR, for
example, tasks talk to the AM and report their individual progress and
counters back to it, via a specific protocol (over the network),
giving the AM more data to compute an near-accurate global progress.

On Fri, Sep 20, 2013 at 12:18 AM, John Lilley <[EMAIL PROTECTED]> wrote:
> How does a YARN application master typically query ongoing status (like
> percentage completion) of its tasks?
>
> I would like to be able to ultimately relay information to the user like:
>
> 100 tasks are scheduled
>
> 10 tasks are complete
>
> 4 tasks are running and they are (4%, 10%, 50%, 70%) complete
>
> But, given that YARN tasks are simply executables, how can the AM even get
> at this information?  Can the AM get access to stdout/stderr?
>
> Thanks
>
> John
>
>

--
Harsh J
+
John Lilley 2013-09-20, 12:02
+
Harsh J 2013-09-20, 13:47
+
John Lilley 2013-09-21, 13:52
+
Arun C Murthy 2013-09-23, 23:41