Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> HDFS Federation Cluster IDs not matched


+
Antoine Boudot 2012-07-25, 12:01
Copy link to this message
-
Re: HDFS Federation Cluster IDs not matched
Antoine,

In order to setup Federation, all the NameNodes have to be first formatted
(or upgraded) with the same cluster ID. This is described here:
http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/Federation.html#Formatting_Namenodes
.

-Vinithra

On Wed, Jul 25, 2012 at 5:01 AM, Antoine Boudot <[EMAIL PROTECTED]> wrote:

>  hello I'm beginner on technology hadoop and I encounter a problem ,****
>
> I am testing to  set 2 namenode of 2 separate host with HDFS federation
> and a pool block  is not to create to startup namenode****
>
> log error here is the datanode****
>
> ** **
>
> 2012-07-25 11:20:39,818 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Storage directory /couloir1/tmp/dfs/data is not formatted.****
>
> 2012-07-25 11:20:39,818 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Formatting ...****
>
> 2012-07-25 11:20:39,858 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Locking is disabled****
>
> 2012-07-25 11:20:39,858 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Storage directory
> /couloir1/tmp/dfs/data/current/BP-459972778-172.31.6.11-1343208016681 is
> not formatted.****
>
> 2012-07-25 11:20:39,858 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Formatting ...****
>
> 2012-07-25 11:20:39,858 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Formatting block pool BP-459972778-172.31.6.11-1343208016681 directory
> /couloir1/tmp/dfs/data/current/BP-459972778-172.31.6.11-1343208016681/current
> ****
>
> 2012-07-25 11:20:39,860 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: Setting up storage:
> nsid=91302135;bpid=BP-459972778-172.31.6.11-1343208016681;lv=-40;nsInfo=lv=-40;cid=CID-345eacf9-d084-4d24-84f0-8294ca7174d8;nsid=91302135;c=0;bpid=BP-459972778-172.31.6.11-1343208016681
> ****
>
> 2012-07-25 11:20:39,870 INFO
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Added
> volume - /couloir1/tmp/dfs/data/current****
>
> 2012-07-25 11:20:39,876 INFO
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl:
> Registered FSDatasetState MBean****
>
> 2012-07-25 11:20:39,880 INFO
> org.apache.hadoop.hdfs.server.datanode.DirectoryScanner: Periodic Directory
> Tree Verification scan starting at 1343224800880 with interval 21600000***
> *
>
> 2012-07-25 11:20:39,881 INFO
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Adding
> block pool BP-459972778-172.31.6.11-1343208016681****
>
> 2012-07-25 11:20:39,894 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool
> BP-459972778-172.31.6.11-1343208016681 (storage id
> DS-1163140142-172.31.6.10-50010-1343208039823) service to hadoop-n10/
> 172.31.6.11:33333 beginning handshake with NN****
>
> 2012-07-25 11:20:39,899 FATAL
> org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for
> block pool Block pool BP-321278605-172.31.4.59-1343207919699 (storage id
> DS-1163140142-172.31.6.10-50010-1343208039823) service to hadoop-vm03/
> 172.31.4.59:33333****
>
> java.io.IOException: Cluster IDs not matched: dn
> cid=CID-345eacf9-d084-4d24-84f0-8294ca7174d8 but ns
> cid=CID-9f7751ec-615b-4f3b-8644-5583ba263a0e;
> bpid=BP-321278605-172.31.4.59-1343207919699****
>
>         at
> org.apache.hadoop.hdfs.server.datanode.DataNode.setClusterId(DataNode.java:307)
> ****
>
>         at
> org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:801)
> ****
>
>         at
> org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:308)
> ****
>
>         at
> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:217)
> ****
>
>         at
> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:660)
> ****
>
>         at java.lang.Thread.run(Thread.java:662)****
>
> 2012-07-25 11:20:39,899 WARN
> org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service
> for: Block pool BP-321278605-172.31.4.59-1343207919699 (storage id
+
Antoine Boudot 2012-07-27, 13:20
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB