Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: Physically moving HDFS cluster to new


Copy link to this message
-
Re: Physically moving HDFS cluster to new
It may or may not help you in your current distress, but MapR's
distribution could handle this pretty easily.

One method is direct distcp between clusters, but you could also use MapR's
mirroring capabilities to migrate data.

You can also carry a MapR cluster, change the IP addresses and relight the
cluster without data loss.  You can also move disks (respecting RAID-0 disk
groups, of course) from machine to machine within a cluster and have them
wake up with all file and directory meta-data intact.

Furthermore, you can lose any two machines in a cluster and are guaranteed
to be able to reconstruct the cluster.  Even if you lose all three replicas
of *any* of the data or meta-data in the cluster, you can *still*
reconstruct any data volumes for which at least one copy survives.  The the
lost data volumes come back at a later time, you will also be able
resurrect the data correctly.

None of this is true for any of the other major Hadoop distributions.

Let me know if you want to try this out.

On Wed, Apr 17, 2013 at 5:20 PM, Tom Brown <[EMAIL PROTECTED]> wrote:

> We have a situation where we want to physically move our small (4 node)
> cluster from one data center to another. As part of this move, each node
> will receive both a new FQN and a new IP address. As I understand it, HDFS
> is somehow tied to the the FQN or IP address, and changing them causes data
> loss.
>
> Is there any supported method of moving a cluster this way?
>
> Thanks in advance!
>
> --Tom
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB