Let's say there is 1PB of data in HDFS and the requirement is to look for a particular sequence in the whole data. If the sequence is found then the MR job could be aborted. There is no meaning in keeping the Map/Reduce tasks running.
There was a query in one of the forums on how a MR job can be aborted if a particular condition is met.
On Sat, Oct 1, 2011 at 9:17 AM, Arun Murthy <[EMAIL PROTECTED]> wrote:
> Not sure what you are trying, or if MR is the right tool... > > Anyway, the basic idea is to write a map-only job with a custom > RecordReader to stop emitting inputs to your map when the > mapper/recordreader finds the needle in the haystack. > > Arun > > Sent from my iPhone > > On Sep 30, 2011, at 8:39 PM, Praveen Sripati <[EMAIL PROTECTED]> > wrote: > > Hi, > > Is there a way to stop an entire job when a certain condition is met in the > map/reduce function? Like looking for a particular key or value. > > Thanks, > Praveen > >
All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by Sematext