Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Submitting a Hadoop task from withing a reducer


Copy link to this message
-
Submitting a Hadoop task from withing a reducer
Hi,
Hadoop tasks are always stacked to form a linear user-managed workflow (a
reduce step cannot start before all previous mappers have stopped etc). This
may be problematic in recursive tasks: for example in a BFS we will not get
any output until the longest branch has been reached.
In order to solve than, an idea came up of submitting a whole Hadoop task
from within a reducer. Have anyone tried it?
Thanks.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB