Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # dev - Doubt regarding use of HTTP during Shuffle phase


+
Pavan Kulkarni 2012-06-27, 16:30
+
Pavan Kulkarni 2012-06-27, 16:33
Copy link to this message
-
Re: Doubt regarding use of HTTP during Shuffle phase
Owen O'Malley 2012-06-27, 16:40
On Wed, Jun 27, 2012 at 9:33 AM, Pavan Kulkarni <[EMAIL PROTECTED]>wrote:

Why is HTTP used during the Shuffle phase
> instead of just creating a Hardlink?
>

The reduces run on a separate machine that the map. Hadoop doesn't use a
shared NFS file system between the machines, because it doesn't scale.

-- Owen
+
Pavan Kulkarni 2012-06-27, 16:44
+
Roman Shaposhnik 2012-06-27, 16:52
+
Pavan Kulkarni 2012-06-27, 16:57
+
Owen OMalley 2012-06-27, 17:21
+
Pavan Kulkarni 2012-06-27, 17:30