-Re: How HDFS decides where to put the block
Nan Zhu 2011-04-18, 15:15
Yes, that's it
On Mon, Apr 18, 2011 at 10:16 PM, Harsh J <[EMAIL PROTECTED]> wrote:
> On Mon, Apr 18, 2011 at 7:16 PM, Nan Zhu <[EMAIL PROTECTED]> wrote:
> > Hi, all
> > I'm confused by a question that "how does the HDFS decide where to put
> > data blocks "
> > I mean that the user invokes some commands like "./hadoop put ***", we
> > assume that this file consistes of 3 blocks, but how HDFS decides where
> > these 3 blocks to be put?
> > Most of the materials don't involve this issue, but just introduce the
> > replica where talking about blocks in HDFS,
> I'm guessing you're looking for the BlockPlacementPolicy
> implementations  and how it is applied in the HDFS.
> Basically, the NameNode chooses the set of DNs for every new-block
> request (from a client) using this policy, and the DFSClient gets a
> list of all the nodes. It goes on to pick the first one among them to
> write the data to. The replication happens async, later.
>  - BlockPolicyPlacementDefault is the default implementation in
> use. It's source available in the o.a.h.hdfs.server.namenode package.
> Harsh J
School of Software,5501
Shanghai Jiao Tong University
E-Mail: [EMAIL PROTECTED]