Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Getting job information, joining a job in hadoop 0.20.1


Copy link to this message
-
Re: Getting job information, joining a job in hadoop 0.20.1
I found give, counters as obtained from RunningJob, I can still use it with
the new style counters.
But I'm still wondering how to get a RunningJob from a JobId.
Is my Hadoop old?

Regards
Saptarshi
On Wed, May 26, 2010 at 2:27 PM, Saptarshi Guha <[EMAIL PROTECTED]>wrote:

> Hello,
> I'm using Hadoop 0.20.1, I submitted a job using the
> org.apache.hadoop.mapreduce.Job approach e.g.
>
> org.apache.hadoop.mapreduce.Job _job
> job_.submit();
>
> However, I would like to,from another Java program, monitor this job:
>
> 1) get map/reduce progress
> 2) get counters
> 3) status
> 4) and the ability to join i.e wait for completion.
>
> say jd = "job_201003190003_1855"
>
> org.apache.hadoop.mapreduce.JobID jid > org.apache.hadoop.mapreduce.JobID.forName(jd);
> org.apache.hadoop.mapred.JobClient jclient = new
> org.apache.hadoop.mapred.JobClient();
>
> Now, what?
>
> http://hadoop.apache.org/common/docs/r0.20.1/api/org/apache/hadoop/mapred/JobClient.html#getJob(org.apache.hadoop.mapred.JobID)
>
> says, I should use
> 1. public RunningJob getJob(JobID jobid)
>
> but, in my installation of hadoop 0.20.1, i do not have such a method.
>
> 2. Even if I could, runningjob.getCounters() returns the old deprecated
> Counters. Is this compatible with the new mapreduce code?
>
> Is there any org.apache.mapreduce  code that given the JobId, I can monitor
> and wait for completion for the job?
>
> I can get a org.apache.hadoop.mapreduce.JobContext with the Job ID, but
> there should be a way to get the Job.
>
> One option, is to serialize the job_ variable and load it in later(not my
> first choice)
> Thanks
>
> State: RUNNING
> Started: Fri Mar 19 00:03:40 EDT 2010
> Version: 0.20.1, r810220
> Compiled: Tue Sep 1 20:55:56 UTC 2009 by oom
> Identifier: 201003190003
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB