Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: are we able to decommission multi nodes at one time?


Copy link to this message
-
Re: are we able to decommission multi nodes at one time?
Azuryy Yu 2013-04-03, 01:53
bq. then namenode start to copy block replicates on DN-2 to another DN,
supposed DN-2.

sorry for typo.

Correct for it:
then namenode start to copy block replicates on DN-1 to another DN,
supposed DN-2.
On Wed, Apr 3, 2013 at 9:51 AM, Azuryy Yu <[EMAIL PROTECTED]> wrote:

> It's different.
> If you just want to stop DN-1 a short time, just kill the DataNode process
> on DN-1. then do what you want. during this time, Namenode  cannot receive
> the heart beat from DN-1, then namenode start to copy block replicates on
> DN-2 to another DN, supposed DN-2.
>
> But when you start DN-1 again, Namenode receive the DN-1 registration,
> then namenode stop to copy the DN-1's block replicates even if NN doesn't
> finish coping.
>
> Am I explain clearly?
>
>
>
> On Wed, Apr 3, 2013 at 9:43 AM, Henry Junyoung Kim <[EMAIL PROTECTED]>wrote:
>
>> @Harsh
>>
>> What's the reasons to make big gaps for removing nodes between
>> decommission and just down nodes?
>> In my understanding, both are necessary to copy un-replicated blocks to
>> another alive nodes.
>> If main costs of  them are this one, total elapsed time couldn't be big
>> different.
>>
>> Could you share some articles or documents to understand about
>> decommissioning procedures?
>> - explaining is always thanks ;)
>>
>>
>> 2013. 4. 2., 오후 5:37, Harsh J <[EMAIL PROTECTED]> 작성:
>>
>> > Yes, you can do the downtime work in steps of 2 DNs at a time,
>> > especially since you mentioned the total work would be only ~30mins at
>> > most.
>> >
>> > On Tue, Apr 2, 2013 at 1:46 PM, Henry Junyoung Kim
>> > <[EMAIL PROTECTED]> wrote:
>> >> the rest of nodes to be alive has enough size to store.
>> >>
>> >> for this one that you've mentioned.
>> >>> its easier to do so in a rolling manner without need of a
>> >>> decommission.
>> >>
>> >> to check my understanding, just shutting down 2 of them and then 2
>> more and then 2 more without decommissions.
>> >>
>> >> is this correct?
>> >>
>> >>
>> >> 2013. 4. 2., 오후 4:54, Harsh J <[EMAIL PROTECTED]> 작성:
>> >>
>> >>> Note though that its only possible to decommission 7 nodes at the same
>> >>> time and expect it to finish iff the remaining 8 nodes have adequate
>> >>> free space for the excess replicas.
>> >>>
>> >>> If you're just going to take them down for a short while (few mins
>> >>> each), its easier to do so in a rolling manner without need of a
>> >>> decommission. You can take upto two down at a time on a replication
>> >>> average of 3 or 3+, and put it back in later without too much data
>> >>> movement impact.
>> >>>
>> >>> On Tue, Apr 2, 2013 at 1:06 PM, Yanbo Liang <[EMAIL PROTECTED]>
>> wrote:
>> >>>> It's reasonable to decommission 7 nodes at the same time.
>> >>>> But may be it also takes long time to finish it.
>> >>>> Because all the replicas in these 7 nodes need to be copied to
>> remaining 8
>> >>>> nodes.
>> >>>> The size of transfer from these nodes to the remaining nodes is
>> equal.
>> >>>>
>> >>>>
>> >>>> 2013/4/2 Henry Junyoung Kim <[EMAIL PROTECTED]>
>> >>>>>
>> >>>>> :)
>> >>>>>
>> >>>>> currently, I  have 15 data nodes.
>> >>>>> for some tests, I am trying to decommission until 8 nodes.
>> >>>>>
>> >>>>> Now, the total dfs used size is 52 TB which is including all
>> replicated
>> >>>>> blocks.
>> >>>>> from 15 to 8, total spent time is almost 4 days long. ;(
>> >>>>>
>> >>>>> someone mentioned that I don't need to decommission node by node.
>> >>>>> for this case, is there no problems if I decommissioned 7 nodes at
>> the
>> >>>>> same time?
>> >>>>>
>> >>>>>
>> >>>>> 2013. 4. 2., 오후 12:14, Azuryy Yu <[EMAIL PROTECTED]> 작성:
>> >>>>>
>> >>>>> I can translate it to native English: how many nodes you want to
>> >>>>> decommission?
>> >>>>>
>> >>>>>
>> >>>>> On Tue, Apr 2, 2013 at 11:01 AM, Yanbo Liang <[EMAIL PROTECTED]>
>> wrote:
>> >>>>>>
>> >>>>>> You want to decommission how many nodes?
>> >>>>>>
>> >>>>>>
>> >>>>>> 2013/4/2 Henry JunYoung KIM <[EMAIL PROTECTED]>
>> >>>>>>>
>> >>>>>>> 15 for datanodes and 3 for replication factor.