Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Getting job information, joining a job in hadoop 0.20.1


Copy link to this message
-
Getting job information, joining a job in hadoop 0.20.1
Hello,
I'm using Hadoop 0.20.1, I submitted a job using the
org.apache.hadoop.mapreduce.Job approach e.g.

org.apache.hadoop.mapreduce.Job _job
job_.submit();

However, I would like to,from another Java program, monitor this job:

1) get map/reduce progress
2) get counters
3) status
4) and the ability to join i.e wait for completion.

say jd = "job_201003190003_1855"

org.apache.hadoop.mapreduce.JobID jid org.apache.hadoop.mapreduce.JobID.forName(jd);
org.apache.hadoop.mapred.JobClient jclient = new
org.apache.hadoop.mapred.JobClient();

Now, what?
http://hadoop.apache.org/common/docs/r0.20.1/api/org/apache/hadoop/mapred/JobClient.html#getJob(org.apache.hadoop.mapred.JobID)

says, I should use
1. public RunningJob getJob(JobID jobid)

but, in my installation of hadoop 0.20.1, i do not have such a method.

2. Even if I could, runningjob.getCounters() returns the old deprecated
Counters. Is this compatible with the new mapreduce code?

Is there any org.apache.mapreduce  code that given the JobId, I can monitor
and wait for completion for the job?

I can get a org.apache.hadoop.mapreduce.JobContext with the Job ID, but
there should be a way to get the Job.

One option, is to serialize the job_ variable and load it in later(not my
first choice)
Thanks

State: RUNNING
Started: Fri Mar 19 00:03:40 EDT 2010
Version: 0.20.1, r810220
Compiled: Tue Sep 1 20:55:56 UTC 2009 by oom
Identifier: 201003190003
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB