Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> Doubt regarding use of HTTP during Shuffle phase


Copy link to this message
-
Re: Doubt regarding use of HTTP during Shuffle phase
On Wed, Jun 27, 2012 at 9:33 AM, Pavan Kulkarni <[EMAIL PROTECTED]>wrote:

Why is HTTP used during the Shuffle phase
> instead of just creating a Hardlink?
>

The reduces run on a separate machine that the map. Hadoop doesn't use a
shared NFS file system between the machines, because it doesn't scale.

-- Owen
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB