Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: are we able to decommission multi nodes at one time?


Copy link to this message
-
Re: are we able to decommission multi nodes at one time?
Azuryy Yu 2013-04-03, 08:18
not at all.  so don't worry about that.
On Wed, Apr 3, 2013 at 2:04 PM, Yanbo Liang <[EMAIL PROTECTED]> wrote:

> It means that may be some replicas will be stay in under replica state?
>
>
> 2013/4/3 Azuryy Yu <[EMAIL PROTECTED]>
>
>> bq. then namenode start to copy block replicates on DN-2 to another DN,
>> supposed DN-2.
>>
>> sorry for typo.
>>
>> Correct for it:
>> then namenode start to copy block replicates on DN-1 to another DN,
>> supposed DN-2.
>>
>>
>> On Wed, Apr 3, 2013 at 9:51 AM, Azuryy Yu <[EMAIL PROTECTED]> wrote:
>>
>>> It's different.
>>> If you just want to stop DN-1 a short time, just kill the DataNode
>>> process on DN-1. then do what you want. during this time, Namenode  cannot
>>> receive the heart beat from DN-1, then namenode start to copy block
>>> replicates on DN-2 to another DN, supposed DN-2.
>>>
>>> But when you start DN-1 again, Namenode receive the DN-1 registration,
>>> then namenode stop to copy the DN-1's block replicates even if NN doesn't
>>> finish coping.
>>>
>>> Am I explain clearly?
>>>
>>>
>>>
>>> On Wed, Apr 3, 2013 at 9:43 AM, Henry Junyoung Kim <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> @Harsh
>>>>
>>>> What's the reasons to make big gaps for removing nodes between
>>>> decommission and just down nodes?
>>>> In my understanding, both are necessary to copy un-replicated blocks to
>>>> another alive nodes.
>>>> If main costs of  them are this one, total elapsed time couldn't be big
>>>> different.
>>>>
>>>> Could you share some articles or documents to understand about
>>>> decommissioning procedures?
>>>> - explaining is always thanks ;)
>>>>
>>>>
>>>> 2013. 4. 2., 오후 5:37, Harsh J <[EMAIL PROTECTED]> 작성:
>>>>
>>>> > Yes, you can do the downtime work in steps of 2 DNs at a time,
>>>> > especially since you mentioned the total work would be only ~30mins at
>>>> > most.
>>>> >
>>>> > On Tue, Apr 2, 2013 at 1:46 PM, Henry Junyoung Kim
>>>> > <[EMAIL PROTECTED]> wrote:
>>>> >> the rest of nodes to be alive has enough size to store.
>>>> >>
>>>> >> for this one that you've mentioned.
>>>> >>> its easier to do so in a rolling manner without need of a
>>>> >>> decommission.
>>>> >>
>>>> >> to check my understanding, just shutting down 2 of them and then 2
>>>> more and then 2 more without decommissions.
>>>> >>
>>>> >> is this correct?
>>>> >>
>>>> >>
>>>> >> 2013. 4. 2., 오후 4:54, Harsh J <[EMAIL PROTECTED]> 작성:
>>>> >>
>>>> >>> Note though that its only possible to decommission 7 nodes at the
>>>> same
>>>> >>> time and expect it to finish iff the remaining 8 nodes have adequate
>>>> >>> free space for the excess replicas.
>>>> >>>
>>>> >>> If you're just going to take them down for a short while (few mins
>>>> >>> each), its easier to do so in a rolling manner without need of a
>>>> >>> decommission. You can take upto two down at a time on a replication
>>>> >>> average of 3 or 3+, and put it back in later without too much data
>>>> >>> movement impact.
>>>> >>>
>>>> >>> On Tue, Apr 2, 2013 at 1:06 PM, Yanbo Liang <[EMAIL PROTECTED]>
>>>> wrote:
>>>> >>>> It's reasonable to decommission 7 nodes at the same time.
>>>> >>>> But may be it also takes long time to finish it.
>>>> >>>> Because all the replicas in these 7 nodes need to be copied to
>>>> remaining 8
>>>> >>>> nodes.
>>>> >>>> The size of transfer from these nodes to the remaining nodes is
>>>> equal.
>>>> >>>>
>>>> >>>>
>>>> >>>> 2013/4/2 Henry Junyoung Kim <[EMAIL PROTECTED]>
>>>> >>>>>
>>>> >>>>> :)
>>>> >>>>>
>>>> >>>>> currently, I  have 15 data nodes.
>>>> >>>>> for some tests, I am trying to decommission until 8 nodes.
>>>> >>>>>
>>>> >>>>> Now, the total dfs used size is 52 TB which is including all
>>>> replicated
>>>> >>>>> blocks.
>>>> >>>>> from 15 to 8, total spent time is almost 4 days long. ;(
>>>> >>>>>
>>>> >>>>> someone mentioned that I don't need to decommission node by node.
>>>> >>>>> for this case, is there no problems if I decommissioned 7 nodes
>>>> at the
>>>> >>>>> same time?
>>>>