Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - new/old APIs


Copy link to this message
-
Re: new/old APIs
Ted Yu 2010-06-21, 16:47
Minor correction:
job Id is in the tracking URL
:50030/jobdetails.jsp?jobid=job_201006171925_0106&refresh=0

On Mon, Jun 21, 2010 at 9:36 AM, Ted Yu <[EMAIL PROTECTED]> wrote:

> In new API, info is a private member of Job. I haven't found an easy way to
> retrieve what you wanted.
>
> You can call getTrackingURL() and try parsing the HTML from the contents of
> HTML. :-)
>
> You can also file a JIRA about this.
>
> FYI mapred is going to be undeprecated. I would suggest spending your time
> on other part of your code.
>
>
>
> On Mon, Jun 21, 2010 at 6:30 AM, Some Body <[EMAIL PROTECTED]>wrote:
>
>> I still haven't figured out how to query the JobTracker for a specific
>> running job using the new API.
>> Is this at all possible? The old/new APIs are driving me crazy.
>>
>> As far as I understand the old API entailed using mapred.JobClient &
>> JobConf
>>
>>  JobConf jobConf = new JobConf(new Configuration(), MyJob.class);
>>  ....
>>  RunningJob runningJob = JobClient.submitJob(jobConf);
>>
>> But in the new API I'm using mapreduce.Job like this:
>>
>>  130: Job job = new Job(new Configuration(), "MyJobName");
>>  131: job.submit();
>>  132: // The job starts ok here but the job.getJobID() gives me a null
>> pointer.
>>  133: System.err.printf("submitted job %s\n", job.getJobID().toString());
>>
>> After submitting the job with job.submit() above I can do this to get all
>> jobs,
>>
>>  100: JobConf jobConf = new JobConf(config, MyDriver.class);
>>  101: JobClient jc = new JobClient(jobConf);
>>  102: JobStatus[] allJobs = jc.getAllJobs();
>>  103: for (int i = 0; i<allJobs.length; i++) {
>>  104:   // check running jobs
>>  105:   int status = allJobs[i].getRunState();
>>  106:     if ( status == 1 ) {
>>  107:       JobID jid = allJobs[i].getJobID();
>>  108:       // compare this id or name to the id/name from my lock file
>>  109:       System.err.printf("job %s is still running\n",
>> jid.toString());
>>  110:     }
>>  111: }
>>
>> but I still need a way to retrieve the job's ID or name after line 131
>> above,
>> because I'd be writing this ID (or name) to a lock file and then killing
>> the
>> job in the loop before starting a new job.
>>
>> Thanks,
>> Alan
>>
>
>