Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Keeping Map-Tasks alive

Copy link to this message
Keeping Map-Tasks alive
Yaron Gonen 2012-08-05, 10:47
Is there a way to keep a map-task alive after it has finished its work, to
later perform another task on its same input?
For example, consider the k-means clustering algorithm (k-means
description<http://en.wikipedia.org/wiki/K-means_clustering>and hadoop
The only thing changing between iterations is the clusters centers. All the
input points remain the same. Keeping the mapper alive, and performing the
next round of map-tasks on the same node will save a lot of communication

Harsh J 2012-08-05, 16:49
Yaron Gonen 2012-08-05, 18:41
Harsh J 2012-08-05, 22:21
Yaron Gonen 2012-08-06, 07:23