Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Shuffle phase replication factor


+
John Lilley 2013-05-21, 18:57
Copy link to this message
-
Re: Shuffle phase replication factor
The map output doesn't get written to HDFS. The map task writes its output to its local disk, the reduce tasks will pull the data through HTTP for further processing.

Am 21.05.2013 um 19:57 schrieb John Lilley <[EMAIL PROTECTED]>:

> When MapReduce enters “shuffle” to partition the tuples, I am assuming that it writes intermediate data to HDFS.  What replication factor is used for those temporary files?
> john
>  

--
Kai Voigt
[EMAIL PROTECTED]
+
John Lilley 2013-05-22, 14:33
+
Shahab Yunus 2013-05-22, 14:37
+
Rahul Bhattacharjee 2013-05-22, 14:51
+
John Lilley 2013-05-22, 14:57
+
Kun Ling 2013-05-23, 01:50
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB