Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Reg: Fetching TaskAttempt Details from a RunningJob


+
Hadoop Learner 2013-01-06, 15:00
Copy link to this message
-
Re: Reg: Fetching TaskAttempt Details from a RunningJob
Hi,

In Hadoop 1.0, I don't think this information is exposed. The
TaskInProgress is an internal class and hence cannot / should not be used
from client applications. The only way out seems to be to screen scrape the
information from the Jobtracker web UI.

If you can live with completed events, then there is something called
TaskCompletionEvents that seem to provide some of this information. You
could look at JobClient.getTaskCompletionEvents.

Please note that in Hadoop 2.0 where mapreduce has been re-architected into
YARN, there are JSON APIs that seem to expose the information you require:

http://hadoop.apache.org/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html#Response_Examples

Look here for "taskAttempts"

Thanks
hemanth
On Sun, Jan 6, 2013 at 8:30 PM, Hadoop Learner
<[EMAIL PROTECTED]>wrote:

> Hi All,
>
>   Working on a requirement of hadoop Job Monitoring. Requirement is to get
> every Task attempt details of a Running Job. Details are following :
>
> Task Attempt Start Time
> Task Tracker Name where Task Attempt is executing
> Errors or Exceptions in a running Task Attempt
>
> I have Implemented JobClient, RunningJob, JobStatus, TaskReport APIs to
> get the Task Attempt IDs( I have the task Attempt IDs for which I need
> required details). By using these APIs I was unable to get Task Attempt
> details as per my requirement mentioned above. Can you please help me with
> some ways to get required details using Java APIs.
>
> Any pointers will be helpful.
>
> (Note : I have tried using TaskInProgress API, But not sure how to get
> TaskInProgress instance from a task ID. Also noticed TaskInProgress class
> is not visible to my classes)
>
> Thanks and Regards,
> Shyam
>