Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> How to stop a MR Job when a certain condition is met?


Copy link to this message
-
Re: How to stop a MR Job when a certain condition is met?
Arun,

Let's say there is 1PB of data in HDFS and the requirement is to look for a
particular sequence in the whole data. If the sequence is found then the MR
job could be aborted. There is no meaning in keeping the Map/Reduce tasks
running.

There was a query in one of the forums on how a MR job can be aborted if a
particular condition is met.

Thanks,
Praveen

On Sat, Oct 1, 2011 at 9:17 AM, Arun Murthy <[EMAIL PROTECTED]> wrote:

> Not sure what you are trying, or if MR is the right tool...
>
> Anyway, the basic idea is to write a map-only job with a custom
> RecordReader to stop emitting inputs to your map when the
> mapper/recordreader finds the needle in the haystack.
>
> Arun
>
> Sent from my iPhone
>
> On Sep 30, 2011, at 8:39 PM, Praveen Sripati <[EMAIL PROTECTED]>
> wrote:
>
> Hi,
>
> Is there a way to stop an entire job when a certain condition is met in the
> map/reduce function? Like looking for a particular key or value.
>
> Thanks,
> Praveen
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB