Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # dev >> Shuffling over the network for local map data.


+
Suresh Kumar 2013-01-22, 03:22
+
Steve Loughran 2013-01-22, 16:46
Copy link to this message
-
Re: Shuffling over the network for local map data.
Hi Steve,

My assumption is that unless it is reading from  http://127.0.0.1/ or
http://localhost/ , it reads over the network. If I'm wrong please correct
me. The http tracker address that a ReduceTask receives is not of that
format. So I do not think it is reading using the loop back address.

Thanks,
Suresh.

On Tue, Jan 22, 2013 at 8:46 AM, Steve Loughran <[EMAIL PROTECTED]>wrote:

> It's just using the loopback address, right -not going on to the external
> network and back again?
>
> On 22 January 2013 03:22, Suresh Kumar <[EMAIL PROTECTED]> wrote:
>
> > Hello,
> >
> > I noticed that the shuffle phase is reading data over http even when data
> > is available locally. The version of hadoop I'm using is 1.0.3. Is there
> a
> > reason it is implemented this way ? Is it OK to make a change that will
> > identify that the data is available locally and read from the local disk
> > instead of the http?
> >
> > I'm new to this developer list and apache developer list in general. So
> > please feel free to let me know if there is a certain etiquette that I'm
> > not following.
> >
> > Thanks,
> > Suresh.
> >
>
+
Suresh Kumar 2013-01-22, 19:02
+
Albert Chu 2013-01-22, 19:42
+
Suresh Kumar 2013-01-22, 23:03
+
Luke Lu 2013-01-22, 19:24
+
Suresh Kumar 2013-01-22, 22:22
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB