Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Supersede a data node help: how to move all files out of a Hadoop data node?


Copy link to this message
-
Re: Supersede a data node help: how to move all files out of a Hadoop data node?
On Fri, Aug 20, 2010 at 4:31 PM, jiang licht <[EMAIL PROTECTED]> wrote:
> Requirement: I want to get rid of a data node machine. But it has useful data that is still in use. So, I want to move all its files/blocks to other live data nodes in the same cluster.
>
> Question: I understand that if a data node is down for a certain amount of time, it will be marked as "dead" and Hadoop will automatically generate a new replication on another live data node for each block on the dead node. So, sooner or later, all files/blocks on the dead data node will be replicated (or "moved") to other data node machines.  My question is:
>
> Is it possible that this process can be explicitly controlled such that I know when all the missing blocks on the dead node are replicated to other live nodes?
>
> What is the recommended way to do this?
>
> How to check there is no missing blocks (or under replication)?
>
> Thanks,
>
> Michael
>
>
>

If you run a 'hadoop fsck /' one of the things reported is Under
replicated blocks. When under replicated blocks =0 everything is
moved. (assuming there are not other problems)