Hi all,

The command bin/hadoop setrepr only affect the metadata in namenode.
So How can I make it take effect on data node ? Thanks.

--
Best Regards

Jeff Zhang

Jeff:

The name node will instruct the data nodes to do either remove or
create new replicas. There's nothing additional you need to do other
than fs -setrep.

On Fri, Jun 11, 2010 at 3:18 AM, Jeff Zhang <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> The command bin/hadoop setrepr only affect the metadata in namenode.
> So How can I make it take effect on data node ? Thanks.
>
> --
> Best Regards
>
> Jeff Zhang
>

--
Eric Sammer
phone: +1-917-287-2675
twitter: esammer
data: www.cloudera.com

But when I invoke command fsck, it show that one replica is missing.
So how can I force the replication ?

On Sat, Jun 12, 2010 at 12:13 PM, Eric Sammer <[EMAIL PROTECTED]> wrote:
> Jeff:
>
> The name node will instruct the data nodes to do either remove or
> create new replicas. There's nothing additional you need to do other
> than fs -setrep.
>
> On Fri, Jun 11, 2010 at 3:18 AM, Jeff Zhang <[EMAIL PROTECTED]> wrote:
>> Hi all,
>>
>> The command bin/hadoop setrepr only affect the metadata in namenode.
>> So How can I make it take effect on data node ? Thanks.
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
> Eric Sammer
> phone: +1-917-287-2675
> twitter: esammer
> data: www.cloudera.com
>

--
Best Regards

Jeff Zhang

There is no way to force replication other than setrep. Do you have
fewer nodes than configured replicas? Hadoop will not place two
replicas on one data node, for instance.

On Sat, Jun 12, 2010 at 12:18 AM, Jeff Zhang <[EMAIL PROTECTED]> wrote:
> But when I invoke command fsck, it show that one replica is missing.
> So how can I force the replication ?
>
> On Sat, Jun 12, 2010 at 12:13 PM, Eric Sammer <[EMAIL PROTECTED]> wrote:
>> Jeff:
>>
>> The name node will instruct the data nodes to do either remove or
>> create new replicas. There's nothing additional you need to do other
>> than fs -setrep.
>>
>> On Fri, Jun 11, 2010 at 3:18 AM, Jeff Zhang <[EMAIL PROTECTED]> wrote:
>>> Hi all,
>>>
>>> The command bin/hadoop setrepr only affect the metadata in namenode.
>>> So How can I make it take effect on data node ? Thanks.
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>>
>>
>> --
>> Eric Sammer
>> phone: +1-917-287-2675
>> twitter: esammer
>> data: www.cloudera.com
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

--
Eric Sammer
phone: +1-917-287-2675
twitter: esammer
data: www.cloudera.com

So when will the replication happen ? Do you know which part of code
control the replication ? Thanks.

On Sun, Jun 13, 2010 at 11:10 AM, Eric Sammer <[EMAIL PROTECTED]> wrote:
> There is no way to force replication other than setrep. Do you have
> fewer nodes than configured replicas? Hadoop will not place two
> replicas on one data node, for instance.
>
> On Sat, Jun 12, 2010 at 12:18 AM, Jeff Zhang <[EMAIL PROTECTED]> wrote:
>> But when I invoke command fsck, it show that one replica is missing.
>> So how can I force the replication ?
>>
>> On Sat, Jun 12, 2010 at 12:13 PM, Eric Sammer <[EMAIL PROTECTED]> wrote:
>>> Jeff:
>>>
>>> The name node will instruct the data nodes to do either remove or
>>> create new replicas. There's nothing additional you need to do other
>>> than fs -setrep.
>>>
>>> On Fri, Jun 11, 2010 at 3:18 AM, Jeff Zhang <[EMAIL PROTECTED]> wrote:
>>>> Hi all,
>>>>
>>>> The command bin/hadoop setrepr only affect the metadata in namenode.
>>>> So How can I make it take effect on data node ? Thanks.
>>>>
>>>> --
>>>> Best Regards
>>>>
>>>> Jeff Zhang
>>>>
>>>
>>>
>>>
>>> --
>>> Eric Sammer
>>> phone: +1-917-287-2675
>>> twitter: esammer
>>> data: www.cloudera.com
>>>
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
> Eric Sammer
> phone: +1-917-287-2675
> twitter: esammer
> data: www.cloudera.com
>

--
Best Regards

Jeff Zhang

On Sat, Jun 12, 2010 at 8:18 PM, Jeff Zhang <[EMAIL PROTECTED]> wrote:

> So when will the replication happen ? Do you know which part of code
> control the replication ? Thanks.
>
>
It's all throughout FSNamesystem and related classes. I wouldn't recommend
digging in there unless you have a couple days to kill - it gets quite
hairy! :)

Replication should happen close to immediately unless there is already a
large queue of underreplicated blocks. Replication happens through a
priority queue, where the priority is based on how far under-replicated a
block is.

-Todd
>
> On Sun, Jun 13, 2010 at 11:10 AM, Eric Sammer <[EMAIL PROTECTED]>
> wrote:
> > There is no way to force replication other than setrep. Do you have
> > fewer nodes than configured replicas? Hadoop will not place two
> > replicas on one data node, for instance.
> >
> > On Sat, Jun 12, 2010 at 12:18 AM, Jeff Zhang <[EMAIL PROTECTED]> wrote:
> >> But when I invoke command fsck, it show that one replica is missing.
> >> So how can I force the replication ?
> >>
> >> On Sat, Jun 12, 2010 at 12:13 PM, Eric Sammer <[EMAIL PROTECTED]>
> wrote:
> >>> Jeff:
> >>>
> >>> The name node will instruct the data nodes to do either remove or
> >>> create new replicas. There's nothing additional you need to do other
> >>> than fs -setrep.
> >>>
> >>> On Fri, Jun 11, 2010 at 3:18 AM, Jeff Zhang <[EMAIL PROTECTED]> wrote:
> >>>> Hi all,
> >>>>
> >>>> The command bin/hadoop setrepr only affect the metadata in namenode.
> >>>> So How can I make it take effect on data node ? Thanks.
> >>>>
> >>>> --
> >>>> Best Regards
> >>>>
> >>>> Jeff Zhang
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Eric Sammer
> >>> phone: +1-917-287-2675
> >>> twitter: esammer
> >>> data: www.cloudera.com
> >>>
> >>
> >>
> >>
> >> --
> >> Best Regards
> >>
> >> Jeff Zhang
> >>
> >
> >
> >
> > --
> > Eric Sammer
> > phone: +1-917-287-2675
> > twitter: esammer
> > data: www.cloudera.com
> >
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

--
Todd Lipcon
Software Engineer, Cloudera