Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> new/old APIs


Copy link to this message
-
Re: new/old APIs
Minor correction:
job Id is in the tracking URL
:50030/jobdetails.jsp?jobid=job_201006171925_0106&refresh=0

On Mon, Jun 21, 2010 at 9:36 AM, Ted Yu <[EMAIL PROTECTED]> wrote:

> In new API, info is a private member of Job. I haven't found an easy way to
> retrieve what you wanted.
>
> You can call getTrackingURL() and try parsing the HTML from the contents of
> HTML. :-)
>
> You can also file a JIRA about this.
>
> FYI mapred is going to be undeprecated. I would suggest spending your time
> on other part of your code.
>
>
>
> On Mon, Jun 21, 2010 at 6:30 AM, Some Body <[EMAIL PROTECTED]>wrote:
>
>> I still haven't figured out how to query the JobTracker for a specific
>> running job using the new API.
>> Is this at all possible? The old/new APIs are driving me crazy.
>>
>> As far as I understand the old API entailed using mapred.JobClient &
>> JobConf
>>
>>  JobConf jobConf = new JobConf(new Configuration(), MyJob.class);
>>  ....
>>  RunningJob runningJob = JobClient.submitJob(jobConf);
>>
>> But in the new API I'm using mapreduce.Job like this:
>>
>>  130: Job job = new Job(new Configuration(), "MyJobName");
>>  131: job.submit();
>>  132: // The job starts ok here but the job.getJobID() gives me a null
>> pointer.
>>  133: System.err.printf("submitted job %s\n", job.getJobID().toString());
>>
>> After submitting the job with job.submit() above I can do this to get all
>> jobs,
>>
>>  100: JobConf jobConf = new JobConf(config, MyDriver.class);
>>  101: JobClient jc = new JobClient(jobConf);
>>  102: JobStatus[] allJobs = jc.getAllJobs();
>>  103: for (int i = 0; i<allJobs.length; i++) {
>>  104:   // check running jobs
>>  105:   int status = allJobs[i].getRunState();
>>  106:     if ( status == 1 ) {
>>  107:       JobID jid = allJobs[i].getJobID();
>>  108:       // compare this id or name to the id/name from my lock file
>>  109:       System.err.printf("job %s is still running\n",
>> jid.toString());
>>  110:     }
>>  111: }
>>
>> but I still need a way to retrieve the job's ID or name after line 131
>> above,
>> because I'd be writing this ID (or name) to a lock file and then killing
>> the
>> job in the loop before starting a new job.
>>
>> Thanks,
>> Alan
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB