Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Re: Hang when add/remove a datanode into/from a 2 datanode cluster


+
sam liu 2013-06-21, 09:43
+
Harsh J 2013-06-21, 13:59
Copy link to this message
-
Hang when add/remove a datanode into/from a 2 datanode cluster
Hi,

I encountered an issue which hangs the decommission operatoin. Its steps:
1. Install a Hadoop 1.1.1 cluster, with 2 datanodes: dn1 and dn2. And, in
hdfs-site.xml, set the 'dfs.replication' to 2
2. Add node dn3 into the cluster as a new datanode, and did not change the '
dfs.replication' value in hdfs-site.xml and keep it as 2
note: step 2 passed
3. Decommission dn3 from the cluster

Expected result: dn3 could be decommissioned successfully

Actual result: decommission progress hangs and the status always be
'Waiting DataNode status: Decommissioned'

However, if the initial cluster includes >= 3 datanodes, this issue won't
be encountered when add/remove another datanode.

Also, after step 2, I noticed that some block's expected replicas is 3, but
the 'dfs.replication' value in hdfs-site.xml is always 2!

Could anyone pls help provide some triages?

Thanks in advance!
+
sam liu 2013-07-31, 06:39
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB