Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Datanode support different Namespace


Copy link to this message
-
Datanode support different Namespace
Hi, all

We try to use hadoop-2.0.5-alpha, using two namespaces, one is for hbase
cluster, and the other one is for common use.At the same time, we use
Quorum Journal policy as HA.

GS-CIX-SEV0001, GS-CIX-SEV0002,  namenodes in hbasecluster  namespace

GS-CIX-SEV0003, GS-CIX-SEV0004, namenodes in commoncluster namespace.

GS-CIX-SEV0001~GS-CIX-SEV0008 , 8 machines used as Datanode

After launching the hdfs cluster all, there is something which makes me
confused, that  each namespace has half of the datanodes.

NameNode 'GS-CIX-SEV0004:9100'

Started:        Thu Jul 04 10:28:00 CST 2013
Version:        2.0.5-alpha, 1488459
Compiled:       2013-06-01T04:05Z by jenkins from branch-2.0.5-alpha
Cluster ID:     CID-15c48d78-2137-4c6e-aacf-0edbf2bb3db7
Block Pool ID:  BP-1792015895-10.100.2.3-1372904504940
Browse the filesystem
NameNode Logs
Go back to DFS home
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Live Datanodes : 4
                Last     Admin    Configured  Used Non DFS  Remaining
Used      Used      Remaining          Block     Block Pool      Failed
     Node      Contact   State     Capacity   (GB)   Used     (GB)
(%)       (%)          (%)    Blocks   Pool      Used (%)>      Volumes
                                     (GB)
(GB)                                                  Used (GB)     Blocks
GS-CIX-SEV0001 1       In Service      888.07 0.00   116.04    772.03 0.00
┌────────────┐     86.93 0           0.00           0.00            0

└────────────┘
GS-CIX-SEV0002 1       In Service      888.07 0.00   135.50    752.57 0.00
┌────────────┐     84.74 0           0.00           0.00            0

└────────────┘
GS-CIX-SEV0005 1       In Service      888.07 0.00    97.61    790.46 0.00
┌────────────┐     89.01 0           0.00           0.00            0

└────────────┘
GS-CIX-SEV0006 1       In Service      888.07 0.00   122.30    765.77 0.00
┌────────────┐     86.23 0           0.00           0.00            0

└────────────┘
Another Namespace's NameNode:

NameNode 'GS-CIX-SEV0001:9100'

Started:        Thu Jul 04 10:19:03 CST 2013
Version:        2.0.5-alpha, 1488459
Compiled:       2013-06-01T04:05Z by jenkins from branch-2.0.5-alpha
Cluster ID:     CID-1a53483d-000e-4726-aef1-f500bedb1df6
Block Pool ID:  BP-1142418822-10.100.2.1-1372904314309
Browse the filesystem
NameNode Logs
Go back to DFS home
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Live Datanodes : 4
                Last     Admin    Configured  Used Non DFS  Remaining
Used      Used      Remaining          Block     Block Pool      Failed
     Node      Contact   State     Capacity   (GB)   Used     (GB)
(%)       (%)          (%)    Blocks   Pool      Used (%)>      Volumes
                                     (GB)
(GB)                                                  Used (GB)     Blocks
GS-CIX-SEV0003 0       In Service      888.07 0.00   150.54    737.53 0.00
┌────────────┐     83.05 0           0.00           0.00            0

└────────────┘
GS-CIX-SEV0004 0       In Service      888.07 0.00   177.22    710.85 0.00
┌────────────┐     80.04 0           0.00           0.00            0

└────────────┘
GS-CIX-SEV0007 0       In Service      888.07 0.00    62.91    825.16 0.00
┌────────────┐     92.92 0           0.00           0.00            0

└────────────┘
GS-CIX-SEV0008 0       In Service      888.07 0.00   125.25    762.82 0.00
┌────────────┐     85.90 0           0.00           0.00            0

And check the DN(GS-CIX-SEV0001)'s log, it prints like this:
2013-07-04 10:34:51,699 FATAL
org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for
block pool Block pool BP-1142418822-10.100.2.1-1372904314309 (storage id
DS-1677272131-10.100.2.1-50010-1372905291690) service to GS-CIX-SEV0001/
10.100.2.1:9100
java.io.IOException: Inconsistent storage IDs. Name-node returned
DS811369792. Expecting DS-1677272131-10.100.2.1-50010-1372905291690
        at
org.apache.hadoop.hdfs.server.datanode.DataNode.bpRegistrationSucceeded(DataNode.java:731)
        at
org.apache.hadoop.hdfs.server.datanode.BPOfferService.registrationSucceeded(BPOfferService.java:308)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.register(BPServiceActor.java:632)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:225)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:664)
        at java.lang.Thread.run(Thread.java:662)

It is proved that one datanode has been required to attached to only one
namespace?

Any views about it will be thankful.

Regards~
Bing Jiang
Tel:(86)134-2619-1361
weibo: http://weibo.com/jiangbinglover
BLOG: http://blog.sina.com.cn/jiangbinglover
National Research Center for Intelligent Computing Systems
Institute of Computing technology
Graduate University of Chinese Academy of Science
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB