Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - HDFS Federation Cluster IDs not matched


+
Antoine Boudot 2012-07-25, 12:01
Copy link to this message
-
Re: HDFS Federation Cluster IDs not matched
Vinithra Varadharajan 2012-07-25, 17:43
Antoine,

In order to setup Federation, all the NameNodes have to be first formatted
(or upgraded) with the same cluster ID. This is described here:
http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/Federation.html#Formatting_Namenodes
.

-Vinithra

On Wed, Jul 25, 2012 at 5:01 AM, Antoine Boudot <[EMAIL PROTECTED]> wrote:

>  hello I'm beginner on technology hadoop and I encounter a problem ,****
>
> I am testing to  set 2 namenode of 2 separate host with HDFS federation
> and a pool block  is not to create to startup namenode****
>
> log error here is the datanode****
>
> ** **
>
> 2012-07-25 11:20:39,818 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Storage directory /couloir1/tmp/dfs/data is not formatted.****
>
> 2012-07-25 11:20:39,818 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Formatting ...****
>
> 2012-07-25 11:20:39,858 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Locking is disabled****
>
> 2012-07-25 11:20:39,858 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Storage directory
> /couloir1/tmp/dfs/data/current/BP-459972778-172.31.6.11-1343208016681 is
> not formatted.****
>
> 2012-07-25 11:20:39,858 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Formatting ...****
>
> 2012-07-25 11:20:39,858 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Formatting block pool BP-459972778-172.31.6.11-1343208016681 directory
> /couloir1/tmp/dfs/data/current/BP-459972778-172.31.6.11-1343208016681/current
> ****
>
> 2012-07-25 11:20:39,860 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: Setting up storage:
> nsid=91302135;bpid=BP-459972778-172.31.6.11-1343208016681;lv=-40;nsInfo=lv=-40;cid=CID-345eacf9-d084-4d24-84f0-8294ca7174d8;nsid=91302135;c=0;bpid=BP-459972778-172.31.6.11-1343208016681
> ****
>
> 2012-07-25 11:20:39,870 INFO
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Added
> volume - /couloir1/tmp/dfs/data/current****
>
> 2012-07-25 11:20:39,876 INFO
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl:
> Registered FSDatasetState MBean****
>
> 2012-07-25 11:20:39,880 INFO
> org.apache.hadoop.hdfs.server.datanode.DirectoryScanner: Periodic Directory
> Tree Verification scan starting at 1343224800880 with interval 21600000***
> *
>
> 2012-07-25 11:20:39,881 INFO
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Adding
> block pool BP-459972778-172.31.6.11-1343208016681****
>
> 2012-07-25 11:20:39,894 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool
> BP-459972778-172.31.6.11-1343208016681 (storage id
> DS-1163140142-172.31.6.10-50010-1343208039823) service to hadoop-n10/
> 172.31.6.11:33333 beginning handshake with NN****
>
> 2012-07-25 11:20:39,899 FATAL
> org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for
> block pool Block pool BP-321278605-172.31.4.59-1343207919699 (storage id
> DS-1163140142-172.31.6.10-50010-1343208039823) service to hadoop-vm03/
> 172.31.4.59:33333****
>
> java.io.IOException: Cluster IDs not matched: dn
> cid=CID-345eacf9-d084-4d24-84f0-8294ca7174d8 but ns
> cid=CID-9f7751ec-615b-4f3b-8644-5583ba263a0e;
> bpid=BP-321278605-172.31.4.59-1343207919699****
>
>         at
> org.apache.hadoop.hdfs.server.datanode.DataNode.setClusterId(DataNode.java:307)
> ****
>
>         at
> org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:801)
> ****
>
>         at
> org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:308)
> ****
>
>         at
> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:217)
> ****
>
>         at
> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:660)
> ****
>
>         at java.lang.Thread.run(Thread.java:662)****
>
> 2012-07-25 11:20:39,899 WARN
> org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service
> for: Block pool BP-321278605-172.31.4.59-1343207919699 (storage id
+
Antoine Boudot 2012-07-27, 13:20