Hi!

I have unbalanced disks on one of my nodes after a disk failure with
disk replacements.

http://wiki.apache.org/hadoop/FAQ#On_an_individual_data_node.2C_how_do_you_balance_the_blocks_on_the_disk.3F
has a solution, but it starts with "Take down HDFS".

Is that really necessary - shouldn't taking down just that datanode,
moving around the blocks, then start the datanode be good enough, or
will that mess up some datastructure in the namenode?

Thanks,
\EF
--
Erik Forsberg <[EMAIL PROTECTED]>
Developer, Opera Software - http://www.opera.com/

On Dec 16, 2010, at 6:34 AM, Erik Forsberg <[EMAIL PROTECTED]> wrote:

> Hi!
>
> I have unbalanced disks on one of my nodes after a disk failure with
> disk replacements.
>
> http://wiki.apache.org/hadoop/FAQ#On_an_individual_data_node.2C_how_do_you_balance_the_blocks_on_the_disk.3F
> has a solution, but it starts with "Take down HDFS".
>
> Is that really necessary - shouldn't taking down just that datanode,
> moving around the blocks, then start the datanode be good enough, or
> will that mess up some datastructure in the namenode?
>

You're right - that should say take the DataNode down, move blocks
around, then restart the DataNode. You might be able to just replace
the disk and have them unbalanced for a while though. As data is
created and deleted things will naturally even out.

--Travis

> Thanks,
> \EF
> --
> Erik Forsberg <[EMAIL PROTECTED]>
> Developer, Opera Software - http://www.opera.com/

On Dec 16, 2010, at 6:32 AM, Erik Forsberg wrote:
> http://wiki.apache.org/hadoop/FAQ#On_an_individual_data_node.2C_how_do_you_balance_the_blocks_on_the_disk.3F
> has a solution, but it starts with "Take down HDFS".
>
> Is that really necessary - shouldn't taking down just that datanode,
> moving around the blocks, then start the datanode be good enough, or
> will that mess up some datastructure in the namenode?
>

It won't mess up the namenode, but it is going to be busy replicating those blocks on that datanode while you move stuff around.  Depending upon how much data is involved, you might find that after you bring the datanode back up the namenode will put you back in a state of major unbalance when it sends a wave of deletions for the extra replicas.