Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Re: Shuffle phase replication factor


Copy link to this message
-
Re: Shuffle phase replication factor
Intermediate data is written to local disk, not to HDFS.

Ian.

On May 21, 2013, at 1:57 PM, John Lilley <[EMAIL PROTECTED]> wrote:

> When MapReduce enters “shuffle” to partition the tuples, I am assuming that it writes intermediate data to HDFS.  What replication factor is used for those temporary files?
> john
>  
---
Ian Wrigley
Sr. Curriculum Manager
Cloudera, Inc
Cell: (323) 819 4075

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB