Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Passing messages


Copy link to this message
-
Re: Passing messages

On Dec 19, 2010, at 10:21 AM, Eric wrote:

> I don't know if there is such a thing in Hadoop, I'm guessing not since MapReduce is designed to have independent mappers and reducers.

Yup.

> I'm just suggesting something here: you could write a small server yourself. Say you start your jobs from a work node that has only light load. On that node, you also create something that works like memcache. (maybe you can even use memcache?)

Or just use ZooKeeper.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB