Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> question about preserving data locality in MapReduce with Yarn


Copy link to this message
-
Re: question about preserving data locality in MapReduce with Yarn
Hi Ricky,

The input splits contain the locations of the blocks they cover.  The AM
gets the information from the input splits and submits requests for those
location.  Each container request spans all the replicas that the block is
located on.  Are you interested in something more specific?

-Sandy
On Mon, Oct 28, 2013 at 7:09 PM, ricky lee <[EMAIL PROTECTED]> wrote:

> Well, I thought an application master can somewhat ask where the data
> exist to a namenode.... isn't it true? If it does not know where the data
> reside, does a MapReduce application master specify the resource name as
> "*" which means data locality might not be preserved at all? thx,
>
> r
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB