Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Re: Datanode support different Namespace


Copy link to this message
-
Re: Datanode support different Namespace
Azuryy Yu 2013-07-04, 08:14
It's random.
 On Jul 4, 2013 3:33 PM, "Bing Jiang" <[EMAIL PROTECTED]> wrote:

> If not set cluster id in formatting the Namenode, is there a policy in
> hdfs to guarantee the even of distributing DataNodes into different
> Namespace, or just randomly?
>
>
>
> 2013/7/4 Azuryy Yu <[EMAIL PROTECTED]>
>
>> Additional,
>>
>> If these are two new clusters, then on each namenode, using "hdfs
>> namenode -format -clusterID yourID"
>>
>> But if you want to upgrade these two clusters from NonHA to HA, then
>> using "bin/start-dfs.sh -upgrade -clusterID yourID"
>>
>>
>>
>> On Thu, Jul 4, 2013 at 3:14 PM, Azuryy Yu <[EMAIL PROTECTED]> wrote:
>>
>>>
>>> This is because you don't use the same clusterID. all data nodes and
>>> namenodes should use the same clusterID.
>>>
>>>
>>> On Thu, Jul 4, 2013 at 3:12 PM, Bing Jiang <[EMAIL PROTECTED]>wrote:
>>>
>>>> Hi, all
>>>>
>>>> We try to use hadoop-2.0.5-alpha, using two namespaces, one is for
>>>> hbase cluster, and the other one is for common use.At the same time, we use
>>>> Quorum Journal policy as HA.
>>>>
>>>> GS-CIX-SEV0001, GS-CIX-SEV0002,  namenodes in hbasecluster  namespace
>>>>
>>>> GS-CIX-SEV0003, GS-CIX-SEV0004, namenodes in commoncluster namespace.
>>>>
>>>> GS-CIX-SEV0001~GS-CIX-SEV0008 , 8 machines used as Datanode
>>>>
>>>> After launching the hdfs cluster all, there is something which makes
>>>> me  confused, that  each namespace has half of the datanodes.
>>>>
>>>> NameNode 'GS-CIX-SEV0004:9100'
>>>>
>>>> Started:        Thu Jul 04 10:28:00 CST 2013
>>>> Version:        2.0.5-alpha, 1488459
>>>> Compiled:       2013-06-01T04:05Z by jenkins from branch-2.0.5-alpha
>>>> Cluster ID:     CID-15c48d78-2137-4c6e-aacf-0edbf2bb3db7
>>>> Block Pool ID:  BP-1792015895-10.100.2.3-1372904504940
>>>>
>>>>
>>>> Browse the filesystem
>>>> NameNode Logs
>>>> Go back to DFS home
>>>>
>>>> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
>>>> Live Datanodes : 4
>>>>
>>>>
>>>>                 Last     Admin    Configured  Used Non DFS  Remaining
>>>> Used      Used      Remaining          Block     Block Pool      Failed
>>>>      Node      Contact   State     Capacity   (GB)   Used     (GB)
>>>> (%)       (%)          (%)    Blocks   Pool      Used (%)>      Volumes
>>>>                                      (GB)
>>>> (GB)                                                  Used (GB)     Blocks
>>>> GS-CIX-SEV0001 1       In Service      888.07 0.00   116.04    772.03
>>>> 0.00 ┌────────────┐     86.93 0           0.00           0.00            0
>>>>
>>>> └────────────┘
>>>> GS-CIX-SEV0002 1       In Service      888.07 0.00   135.50    752.57
>>>> 0.00 ┌────────────┐     84.74 0           0.00           0.00            0
>>>>
>>>> └────────────┘
>>>> GS-CIX-SEV0005 1       In Service      888.07 0.00    97.61    790.46
>>>> 0.00 ┌────────────┐     89.01 0           0.00           0.00            0
>>>>
>>>> └────────────┘
>>>> GS-CIX-SEV0006 1       In Service      888.07 0.00   122.30    765.77
>>>> 0.00 ┌────────────┐     86.23 0           0.00           0.00            0
>>>>
>>>> └────────────┘
>>>>
>>>>
>>>> Another Namespace's NameNode:
>>>>
>>>> NameNode 'GS-CIX-SEV0001:9100'
>>>>
>>>> Started:        Thu Jul 04 10:19:03 CST 2013
>>>> Version:        2.0.5-alpha, 1488459
>>>> Compiled:       2013-06-01T04:05Z by jenkins from branch-2.0.5-alpha
>>>> Cluster ID:     CID-1a53483d-000e-4726-aef1-f500bedb1df6
>>>> Block Pool ID:  BP-1142418822-10.100.2.1-1372904314309
>>>>
>>>>
>>>> Browse the filesystem
>>>> NameNode Logs
>>>> Go back to DFS home
>>>>
>>>> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
>>>