Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Supersede a data node help: how to move all files out of a Hadoop data node?


Copy link to this message
-
Re: Supersede a data node help: how to move all files out of a Hadoop data node?
On Fri, Aug 20, 2010 at 4:31 PM, jiang licht <[EMAIL PROTECTED]> wrote:
> Requirement: I want to get rid of a data node machine. But it has useful data that is still in use. So, I want to move all its files/blocks to other live data nodes in the same cluster.
>
> Question: I understand that if a data node is down for a certain amount of time, it will be marked as "dead" and Hadoop will automatically generate a new replication on another live data node for each block on the dead node. So, sooner or later, all files/blocks on the dead data node will be replicated (or "moved") to other data node machines.  My question is:
>
> Is it possible that this process can be explicitly controlled such that I know when all the missing blocks on the dead node are replicated to other live nodes?
>
> What is the recommended way to do this?
>
> How to check there is no missing blocks (or under replication)?
>
> Thanks,
>
> Michael
>
>
>

If you run a 'hadoop fsck /' one of the things reported is Under
replicated blocks. When under replicated blocks =0 everything is
moved. (assuming there are not other problems)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB