Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - question about preserving data locality in MapReduce with Yarn


Copy link to this message
-
Re: question about preserving data locality in MapReduce with Yarn
Sandy Ryza 2013-10-29, 02:21
Hi Ricky,

The input splits contain the locations of the blocks they cover.  The AM
gets the information from the input splits and submits requests for those
location.  Each container request spans all the replicas that the block is
located on.  Are you interested in something more specific?

-Sandy
On Mon, Oct 28, 2013 at 7:09 PM, ricky lee <[EMAIL PROTECTED]> wrote:

> Well, I thought an application master can somewhat ask where the data
> exist to a namenode.... isn't it true? If it does not know where the data
> reside, does a MapReduce application master specify the resource name as
> "*" which means data locality might not be preserved at all? thx,
>
> r
>