Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Getting job information, joining a job in hadoop 0.20.1


Copy link to this message
-
Re: Getting job information, joining a job in hadoop 0.20.1
Hmm,
Oh well, a bit of effort pays off:
On 0.20.2

org.apache.hadoop.mapreduce.JobID jid org.apache.hadoop.mapreduce.JobID.forName(jd);
 org.apache.hadoop.mapred.JobClient jclient = new
org.apache.hadoop.mapred.JobClient(
    org.apache.hadoop.mapred.JobTracker.getAddress(new Configuration()),new
Configuration());
 org.apache.hadoop.mapred.JobID jj org.apache.hadoop.mapred.JobID.downgrade(jid);
org.apache.hadoop.mapred.RunningJob rj = jclient.getJob(jj);
 System.out.println(rj);
On Wed, May 26, 2010 at 2:48 PM, Saptarshi Guha <[EMAIL PROTECTED]>wrote:

> I found give, counters as obtained from RunningJob, I can still use it with
> the new style counters.
> But I'm still wondering how to get a RunningJob from a JobId.
> Is my Hadoop old?
>
> Regards
> Saptarshi
>
>
> On Wed, May 26, 2010 at 2:27 PM, Saptarshi Guha <[EMAIL PROTECTED]>wrote:
>
>> Hello,
>> I'm using Hadoop 0.20.1, I submitted a job using the
>> org.apache.hadoop.mapreduce.Job approach e.g.
>>
>> org.apache.hadoop.mapreduce.Job _job
>> job_.submit();
>>
>> However, I would like to,from another Java program, monitor this job:
>>
>> 1) get map/reduce progress
>> 2) get counters
>> 3) status
>> 4) and the ability to join i.e wait for completion.
>>
>> say jd = "job_201003190003_1855"
>>
>> org.apache.hadoop.mapreduce.JobID jid >> org.apache.hadoop.mapreduce.JobID.forName(jd);
>> org.apache.hadoop.mapred.JobClient jclient = new
>> org.apache.hadoop.mapred.JobClient();
>>
>> Now, what?
>>
>> http://hadoop.apache.org/common/docs/r0.20.1/api/org/apache/hadoop/mapred/JobClient.html#getJob(org.apache.hadoop.mapred.JobID)
>>
>> says, I should use
>> 1. public RunningJob getJob(JobID jobid)
>>
>> but, in my installation of hadoop 0.20.1, i do not have such a method.
>>
>> 2. Even if I could, runningjob.getCounters() returns the old deprecated
>> Counters. Is this compatible with the new mapreduce code?
>>
>> Is there any org.apache.mapreduce  code that given the JobId, I can
>> monitor and wait for completion for the job?
>>
>> I can get a org.apache.hadoop.mapreduce.JobContext with the Job ID, but
>> there should be a way to get the Job.
>>
>> One option, is to serialize the job_ variable and load it in later(not my
>> first choice)
>> Thanks
>>
>> State: RUNNING
>> Started: Fri Mar 19 00:03:40 EDT 2010
>> Version: 0.20.1, r810220
>> Compiled: Tue Sep 1 20:55:56 UTC 2009 by oom
>> Identifier: 201003190003
>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB