Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Passing messages

Copy link to this message
Re: Passing messages
On 12/18/2010 11:04 AM, Martin Becker wrote:
> Hello everbody,
> I am wondering if there is a feature allowing (in my case) reduce
> tasks to communicate. For example by some volatile variables at some
> centralized point. Or maybe just notify other running or to-be-running
> reduce tasks of a completed reduce task featuring some arguments.
> In my case, I have reduce tasks doing computations that will
> output/produce certain quality threshold. Other reduce tasks can/could
> estimate, if they ever get above those thresholds. If not they could
> just cease running.
> Thanks in advance,
> Martin
> PS: If such functions are not yet part of the API, I would like to
> know if there are good reasons for it and if not, propose to introduce
> such functionality.

You could use Zookeeper to do this.