Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> migration from hadoop cluster cdh3 to cdh4


Copy link to this message
-
migration from hadoop cluster cdh3 to cdh4
Hi,

Is there any instructions or documents covering migration from hadoop hdfs
cdh3 to cdh4 since all the docs I found are talking about in place
upgrading ONLY?

I have two hadoop clusters, My target is to use hadoop -cp to copy all the
hdfs files from *cluster1* to*cluster2*

*Cluster1:* Hadoop 0.20.2-cdh3u4

*Cluster2:* Hadoop 2.0.0-cdh4.1.1

Now, even just running dfs -ls command against *cluster1* remotely on *
cluster2* as below:

hadoop fs -ls hdfs://cluster1-namenode:8020/hbase

I am getting the exception:

*ls: Failed on local exception: java.io.IOException: Response is null.;
Host Details : local host is: "cluster2-namenode/10.21.xxx.xxx";
destination host is: "cluster1-namenode":8020;*
I think it's due to the hadoop version difference. In my case, cdh3 cluster
doesn't have mapred deployed which rules out all the distcp, bhase
copytable options. And the hbase replication ability is not available on
cdh3 cluster neither. I am struggling to think of a way to migrate the hdfs
data from *cluster1* to *cluster2.*

--
All the best,
Shengjie Min
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB