Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> How to stop a MR Job when a certain condition is met?

Copy link to this message
Re: How to stop a MR Job when a certain condition is met?

Let's say there is 1PB of data in HDFS and the requirement is to look for a
particular sequence in the whole data. If the sequence is found then the MR
job could be aborted. There is no meaning in keeping the Map/Reduce tasks

There was a query in one of the forums on how a MR job can be aborted if a
particular condition is met.


On Sat, Oct 1, 2011 at 9:17 AM, Arun Murthy <[EMAIL PROTECTED]> wrote:

> Not sure what you are trying, or if MR is the right tool...
> Anyway, the basic idea is to write a map-only job with a custom
> RecordReader to stop emitting inputs to your map when the
> mapper/recordreader finds the needle in the haystack.
> Arun
> Sent from my iPhone
> On Sep 30, 2011, at 8:39 PM, Praveen Sripati <[EMAIL PROTECTED]>
> wrote:
> Hi,
> Is there a way to stop an entire job when a certain condition is met in the
> map/reduce function? Like looking for a particular key or value.
> Thanks,
> Praveen