Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Rack decommission


Copy link to this message
-
Re: Rack decommission
Hi David,

> If a DN is decommissioned and returns, Does the NM update it's block meta
data. If it does,  How does it decide when to update the meta data map?

I presume you meant "the NN update it's block meta data". The answer is
yes. Soon as the NN decides that a node is dead / decommissioned and that
blocks have / will fallen below their replication factor, it will order
more replications. This is what is going to happen when you decommission
the nodes.

When the DNs come back, and they report their blocks to the NN, the NN will
realise that the blocks are over-replicated and will order deletions.

> When I view the "dfsnodelist.jsp?whatNodesDECOMMISSIONING"  page, there are many slaves with "Blocks With No Live
Replicas"

Obviously depends on how loaded your HDFS was.

On Thu, May 3, 2012 at 10:04 AM, David Ginzburg <[EMAIL PROTECTED]> wrote:

>  Hi,
> I have began decommissioning the servers  in the rack.
> When I view the "dfsnodelist.jsp?whatNodes=DECOMMISSIONING"  page, there
> are many slaves with "Blocks With No Live Replicas"
> Is it possible , even if I have rack awareness configured ?
>
>
>
> ------------------------------
> From: [EMAIL PROTECTED]
> To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
> Subject: Rack decommission
> Date: Wed, 2 May 2012 11:49:41 +0000
>
>
>  Hi,
>
> I have a cluster with 4 racks.
> I am scheduled to replace one of them, with slave servers included.
> The cluster is configured with rack awareness and replication factor is 3.
> It is my understanding that on rack aware clusters, block placement policy
> ensures blocks are at least on two racks.
> I am concerned about decommission of whole rack.
>
> I considered decommission each node in the rack separately and let it
> balance automatically  , but I am afraid it will take a lot of time.
>
> If a DN is decommissioned and returns, Does the NM update it's block meta
> data. If it does,  How does it decide when to update the meta data map?
>
> Is there a suggested way to decommission a whole rack , I may have missed?
>
>
>
>
>
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB