Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Re: are we able to decommission multi nodes at one time?


+
Henry Junyoung Kim 2013-04-02, 04:26
+
Yanbo Liang 2013-04-01, 11:17
+
Henry JunYoung KIM 2013-04-02, 01:35
+
Azuryy Yu 2013-04-03, 01:53
Copy link to this message
-
Re: are we able to decommission multi nodes at one time?
Yanbo Liang 2013-04-03, 06:04
It means that may be some replicas will be stay in under replica state?
2013/4/3 Azuryy Yu <[EMAIL PROTECTED]>

> bq. then namenode start to copy block replicates on DN-2 to another DN,
> supposed DN-2.
>
> sorry for typo.
>
> Correct for it:
> then namenode start to copy block replicates on DN-1 to another DN,
> supposed DN-2.
>
>
> On Wed, Apr 3, 2013 at 9:51 AM, Azuryy Yu <[EMAIL PROTECTED]> wrote:
>
>> It's different.
>> If you just want to stop DN-1 a short time, just kill the DataNode
>> process on DN-1. then do what you want. during this time, Namenode  cannot
>> receive the heart beat from DN-1, then namenode start to copy block
>> replicates on DN-2 to another DN, supposed DN-2.
>>
>> But when you start DN-1 again, Namenode receive the DN-1 registration,
>> then namenode stop to copy the DN-1's block replicates even if NN doesn't
>> finish coping.
>>
>> Am I explain clearly?
>>
>>
>>
>> On Wed, Apr 3, 2013 at 9:43 AM, Henry Junyoung Kim <[EMAIL PROTECTED]
>> > wrote:
>>
>>> @Harsh
>>>
>>> What's the reasons to make big gaps for removing nodes between
>>> decommission and just down nodes?
>>> In my understanding, both are necessary to copy un-replicated blocks to
>>> another alive nodes.
>>> If main costs of  them are this one, total elapsed time couldn't be big
>>> different.
>>>
>>> Could you share some articles or documents to understand about
>>> decommissioning procedures?
>>> - explaining is always thanks ;)
>>>
>>>
>>> 2013. 4. 2., 오후 5:37, Harsh J <[EMAIL PROTECTED]> 작성:
>>>
>>> > Yes, you can do the downtime work in steps of 2 DNs at a time,
>>> > especially since you mentioned the total work would be only ~30mins at
>>> > most.
>>> >
>>> > On Tue, Apr 2, 2013 at 1:46 PM, Henry Junyoung Kim
>>> > <[EMAIL PROTECTED]> wrote:
>>> >> the rest of nodes to be alive has enough size to store.
>>> >>
>>> >> for this one that you've mentioned.
>>> >>> its easier to do so in a rolling manner without need of a
>>> >>> decommission.
>>> >>
>>> >> to check my understanding, just shutting down 2 of them and then 2
>>> more and then 2 more without decommissions.
>>> >>
>>> >> is this correct?
>>> >>
>>> >>
>>> >> 2013. 4. 2., 오후 4:54, Harsh J <[EMAIL PROTECTED]> 작성:
>>> >>
>>> >>> Note though that its only possible to decommission 7 nodes at the
>>> same
>>> >>> time and expect it to finish iff the remaining 8 nodes have adequate
>>> >>> free space for the excess replicas.
>>> >>>
>>> >>> If you're just going to take them down for a short while (few mins
>>> >>> each), its easier to do so in a rolling manner without need of a
>>> >>> decommission. You can take upto two down at a time on a replication
>>> >>> average of 3 or 3+, and put it back in later without too much data
>>> >>> movement impact.
>>> >>>
>>> >>> On Tue, Apr 2, 2013 at 1:06 PM, Yanbo Liang <[EMAIL PROTECTED]>
>>> wrote:
>>> >>>> It's reasonable to decommission 7 nodes at the same time.
>>> >>>> But may be it also takes long time to finish it.
>>> >>>> Because all the replicas in these 7 nodes need to be copied to
>>> remaining 8
>>> >>>> nodes.
>>> >>>> The size of transfer from these nodes to the remaining nodes is
>>> equal.
>>> >>>>
>>> >>>>
>>> >>>> 2013/4/2 Henry Junyoung Kim <[EMAIL PROTECTED]>
>>> >>>>>
>>> >>>>> :)
>>> >>>>>
>>> >>>>> currently, I  have 15 data nodes.
>>> >>>>> for some tests, I am trying to decommission until 8 nodes.
>>> >>>>>
>>> >>>>> Now, the total dfs used size is 52 TB which is including all
>>> replicated
>>> >>>>> blocks.
>>> >>>>> from 15 to 8, total spent time is almost 4 days long. ;(
>>> >>>>>
>>> >>>>> someone mentioned that I don't need to decommission node by node.
>>> >>>>> for this case, is there no problems if I decommissioned 7 nodes at
>>> the
>>> >>>>> same time?
>>> >>>>>
>>> >>>>>
>>> >>>>> 2013. 4. 2., 오후 12:14, Azuryy Yu <[EMAIL PROTECTED]> 작성:
>>> >>>>>
>>> >>>>> I can translate it to native English: how many nodes you want to
>>> >>>>> decommission?
>>> >>>>>
>>> >>>>>
>>> >>>>> On Tue, Apr 2, 2013 at 11:01 AM, Yanbo Liang <[EMAIL PROTECTED]>
+
Azuryy Yu 2013-04-03, 08:18