Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Keeping Map-Tasks alive

Copy link to this message
Keeping Map-Tasks alive
Is there a way to keep a map-task alive after it has finished its work, to
later perform another task on its same input?
For example, consider the k-means clustering algorithm (k-means
description<http://en.wikipedia.org/wiki/K-means_clustering>and hadoop
The only thing changing between iterations is the clusters centers. All the
input points remain the same. Keeping the mapper alive, and performing the
next round of map-tasks on the same node will save a lot of communication