Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Unbalanced disks - need to take down whole HDFS?


Copy link to this message
-
Re: Unbalanced disks - need to take down whole HDFS?

On Dec 16, 2010, at 6:32 AM, Erik Forsberg wrote:
> http://wiki.apache.org/hadoop/FAQ#On_an_individual_data_node.2C_how_do_you_balance_the_blocks_on_the_disk.3F
> has a solution, but it starts with "Take down HDFS".
>
> Is that really necessary - shouldn't taking down just that datanode,
> moving around the blocks, then start the datanode be good enough, or
> will that mess up some datastructure in the namenode?
>

It won't mess up the namenode, but it is going to be busy replicating those blocks on that datanode while you move stuff around.  Depending upon how much data is involved, you might find that after you bring the datanode back up the namenode will put you back in a state of major unbalance when it sends a wave of deletions for the extra replicas.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB