Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - Re: Shuffle phase replication factor


Copy link to this message
-
Re: Shuffle phase replication factor
Ian Wrigley 2013-05-21, 18:59
Intermediate data is written to local disk, not to HDFS.

Ian.

On May 21, 2013, at 1:57 PM, John Lilley <[EMAIL PROTECTED]> wrote:

> When MapReduce enters “shuffle” to partition the tuples, I am assuming that it writes intermediate data to HDFS.  What replication factor is used for those temporary files?
> john
>  
---
Ian Wrigley
Sr. Curriculum Manager
Cloudera, Inc
Cell: (323) 819 4075