Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Getting job information, joining a job in hadoop 0.20.1


Copy link to this message
-
Getting job information, joining a job in hadoop 0.20.1
Hello,
I'm using Hadoop 0.20.1, I submitted a job using the
org.apache.hadoop.mapreduce.Job approach e.g.

org.apache.hadoop.mapreduce.Job _job
job_.submit();

However, I would like to,from another Java program, monitor this job:

1) get map/reduce progress
2) get counters
3) status
4) and the ability to join i.e wait for completion.

say jd = "job_201003190003_1855"

org.apache.hadoop.mapreduce.JobID jid org.apache.hadoop.mapreduce.JobID.forName(jd);
org.apache.hadoop.mapred.JobClient jclient = new
org.apache.hadoop.mapred.JobClient();

Now, what?
http://hadoop.apache.org/common/docs/r0.20.1/api/org/apache/hadoop/mapred/JobClient.html#getJob(org.apache.hadoop.mapred.JobID)

says, I should use
1. public RunningJob getJob(JobID jobid)

but, in my installation of hadoop 0.20.1, i do not have such a method.

2. Even if I could, runningjob.getCounters() returns the old deprecated
Counters. Is this compatible with the new mapreduce code?

Is there any org.apache.mapreduce  code that given the JobId, I can monitor
and wait for completion for the job?

I can get a org.apache.hadoop.mapreduce.JobContext with the Job ID, but
there should be a way to get the Job.

One option, is to serialize the job_ variable and load it in later(not my
first choice)
Thanks

State: RUNNING
Started: Fri Mar 19 00:03:40 EDT 2010
Version: 0.20.1, r810220
Compiled: Tue Sep 1 20:55:56 UTC 2009 by oom
Identifier: 201003190003