Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Skipping entire task

Håvard Wahl Kongsgård 2013-01-05, 09:53
Copy link to this message
RE: Skipping entire task
Thinking here... if you submitted the task programmatically you should be
able to capture the failure of the task and gracefully move past it to your
next tasks.

To say it in a long-winded way:  Let's say you submit a job to Hadoop, a
java jar, and your main class implements Tool. That code has the
responsibility to submit a series of jobs to hadoop, something like this:

  Job myJob = new MyJob(getConf());
}catch(Exception uhhohh){
  //Deal with the issue and move on
Job myNextJob = new MyNextJob(getConf());

Just pseudo code there to demonstrate my thought.


-----Original Message-----
From: Håvard Wahl Kongsgård [mailto:[EMAIL PROTECTED]]
Sent: Saturday, January 05, 2013 4:54 PM
To: user
Subject: Skipping entire task

Hi, hadoop can skip bad records
But it is also possible to skip entire tasks?


Håvard Wahl Kongsgård
Faculty of Medicine &
Department of Mathematical Sciences