Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Unbalanced disks - need to take down whole HDFS?

Copy link to this message
Re: Unbalanced disks - need to take down whole HDFS?

On Dec 16, 2010, at 6:32 AM, Erik Forsberg wrote:
> http://wiki.apache.org/hadoop/FAQ#On_an_individual_data_node.2C_how_do_you_balance_the_blocks_on_the_disk.3F
> has a solution, but it starts with "Take down HDFS".
> Is that really necessary - shouldn't taking down just that datanode,
> moving around the blocks, then start the datanode be good enough, or
> will that mess up some datastructure in the namenode?

It won't mess up the namenode, but it is going to be busy replicating those blocks on that datanode while you move stuff around.  Depending upon how much data is involved, you might find that after you bring the datanode back up the namenode will put you back in a state of major unbalance when it sends a wave of deletions for the extra replicas.