Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> Doubt regarding use of HTTP during Shuffle phase


Copy link to this message
-
Re: Doubt regarding use of HTTP during Shuffle phase
On Wed, Jun 27, 2012 at 9:33 AM, Pavan Kulkarni <[EMAIL PROTECTED]>wrote:

Why is HTTP used during the Shuffle phase
> instead of just creating a Hardlink?
>

The reduces run on a separate machine that the map. Hadoop doesn't use a
shared NFS file system between the machines, because it doesn't scale.

-- Owen