Hi,

I am doing some research on the data placement, but I am not quite familiar
with the Hadoop, is there any one who can tell me when a new file be added
into the HDFS, which function will be called by namenode to make a decision
on the allocation ( I mean, if there has a list of datanodes and the file
might be larger than 64MB, that is, several blocks may be included. What's
the function that decide the placement of those blocks to those datanode?)

Thanks for any help you provided!

On Fri, Aug 24, 2012 at 3:33 PM, wei xu <[EMAIL PROTECTED]> wrote:
> I am doing some research on the data placement, but I am not quite familiar
> with the Hadoop, is there any one who can tell me when a new file be added
> into the HDFS, which function will be called by namenode to make a decision
> on the allocation ( I mean, if there has a list of datanodes and the file
> might be larger than 64MB, that is, several blocks may be included. What's
> the function that decide the placement of those blocks to those datanode?)

You'll want to read BlockPlacementPolicy and its inheritors.

-andy

Thanks but I found that BlockPlacementPolicy only appear after 0.21.0? Is
there any similar class or function appeared in 0.20.
On Fri, Aug 24, 2012 at 6:03 PM, Andy Isaacson <[EMAIL PROTECTED]> wrote:

> On Fri, Aug 24, 2012 at 3:33 PM, wei xu <[EMAIL PROTECTED]> wrote:
> > I am doing some research on the data placement, but I am not quite
> familiar
> > with the Hadoop, is there any one who can tell me when a new file be
> added
> > into the HDFS, which function will be called by namenode to make a
> decision
> > on the allocation ( I mean, if there has a list of datanodes and the file
> > might be larger than 64MB, that is, several blocks may be included.
> What's
> > the function that decide the placement of those blocks to those
> datanode?)
>
> You'll want to read BlockPlacementPolicy and its inheritors.
>
> -andy
>

For 0.20 version, you should go to ReplicationTargetChooser which is called
from FSNamesystem.

Le Hieu Hanh
On Sat, Aug 25, 2012 at 9:00 AM, Vivi Lang <[EMAIL PROTECTED]> wrote:

> Thanks but I found that BlockPlacementPolicy only appear after 0.21.0? Is
> there any similar class or function appeared in 0.20.
>
>
> On Fri, Aug 24, 2012 at 6:03 PM, Andy Isaacson <[EMAIL PROTECTED]> wrote:
>
> > On Fri, Aug 24, 2012 at 3:33 PM, wei xu <[EMAIL PROTECTED]> wrote:
> > > I am doing some research on the data placement, but I am not quite
> > familiar
> > > with the Hadoop, is there any one who can tell me when a new file be
> > added
> > > into the HDFS, which function will be called by namenode to make a
> > decision
> > > on the allocation ( I mean, if there has a list of datanodes and the
> file
> > > might be larger than 64MB, that is, several blocks may be included.
> > What's
> > > the function that decide the placement of those blocks to those
> > datanode?)
> >
> > You'll want to read BlockPlacementPolicy and its inheritors.
> >
> > -andy
> >
>