Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # dev >> migration from hadoop cluster cdh3 to cdh4

Copy link to this message
migration from hadoop cluster cdh3 to cdh4

Is there any instructions or documents covering migration from hadoop hdfs
cdh3 to cdh4 since all the docs I found are talking about in place
upgrading ONLY?

I have two hadoop clusters, My target is to use hadoop -cp to copy all the
hdfs files from *cluster1* to*cluster2*

*Cluster1:* Hadoop 0.20.2-cdh3u4

*Cluster2:* Hadoop 2.0.0-cdh4.1.1

Now, even just running dfs -ls command against *cluster1* remotely on *
cluster2* as below:

hadoop fs -ls hdfs://cluster1-namenode:8020/hbase

I am getting the exception:

*ls: Failed on local exception: java.io.IOException: Response is null.;
Host Details : local host is: "cluster2-namenode/10.21.xxx.xxx";
destination host is: "cluster1-namenode":8020;*
I think it's due to the hadoop version difference. In my case, cdh3 cluster
doesn't have mapred deployed which rules out all the distcp, bhase
copytable options. And the hbase replication ability is not available on
cdh3 cluster neither. I am struggling to think of a way to migrate the hdfs
data from *cluster1* to *cluster2.*

All the best,
Shengjie Min
Harsh J 2012-12-07, 13:41