Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: 【dfs.namenode.shared.edits.dir can support different NameServices】


Copy link to this message
-
Re: 【dfs.namenode.shared.edits.dir can support different NameServices】
Thanks for Azuryy's reply.

I used configuration:

<property>
  <name>dfs.namenode.shared.edits.dir.hbasecluster.hnn1</name>

<value>qjournal://GS-CIX-SEV0001:8485;GS-CIX-SEV0002:8485;GS-CIX-SEV0003:8485/hbasecluster</value>
</property>

<property>
 <name>dfs.namenode.shared.edits.dir.hbasecluster.hnn2</name>

<value>qjournal://GS-CIX-SEV0001:8485;GS-CIX-SEV0002:8485;GS-CIX-SEV0003:8485/hbasecluster</value>
</property>
<property>
  <name>dfs.namenode.shared.edits.dir.commoncluster.cnn1</name>

<value>qjournal://GS-CIX-SEV0001:8485;GS-CIX-SEV0002:8485;GS-CIX-SEV0003:8485/commoncluster</value>
</property>
<property>
  <name>dfs.namenode.shared.edits.dir.commoncluster.cnn2</name>

<value>qjournal://GS-CIX-SEV0001:8485;GS-CIX-SEV0002:8485;GS-CIX-SEV0003:8485/commoncluster</value>
</property>

 Start JournalNodes by hands. and everything goes well now.
2013/7/4 Azuryy Yu <[EMAIL PROTECTED]>

> Hi Bing,
> HA is not confilct with HDFS federation.
>
> for example, you have two name services: cluster1, cluster2,
>
> then,
>
> <property>
>     <name>dfs.namenode.shared.edits.dir</name>
>     <value>qjournal://n1.com:8485;n2.com:8485/cluster1</value>
> </property>
>
> <property>
>     <name>dfs.namenode.shared.edits.dir</name>
>     <value>qjournal://n1.com:8485;n2.com:8485/cluster2</value>
>  </property>
>
>
> On Thu, Jul 4, 2013 at 2:46 PM, Bing Jiang <[EMAIL PROTECTED]>wrote:
>
>> hi, Chris.
>> I have traced the source code, and I find this issue comes from
>> sbin/start-dfs.sh,
>>
>> SHARED_EDITS_DIR=$($HADOOP_PREFIX/bin/hdfs getconf -confKey
>> dfs.namenode.shared.edits.dir 2>&-)
>>
>> If I set the suffix on dfs.namenode.shared.edits.dir.[namespace id].[nn
>> id], it will get null. So please take consideration about HA on multiple
>> nameservice.
>> So, please change the way of launching JournalNodes in  shell scripts
>> (sbin/start-dfs.sh)
>>
>>
>> Thanks.
>>
>>
>> 2013/7/4 Chris Nauroth <[EMAIL PROTECTED]>
>>
>>> Hello Bing,
>>>
>>> I have not tested this configuration myself, but from reading the code
>>> it does appear that dfs.namenode.shared.edits.dir supports appending the
>>> nameservice ID to the key, so that you can specify different directories
>>> for different federated namenodes in a single hdfs-site.xml.
>>>
>>>
>>> https://github.com/apache/hadoop-common/blob/branch-2.0.5-alpha/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java#L147
>>>
>>> Hope this helps,
>>>
>>> Chris Nauroth
>>> Hortonworks
>>> http://hortonworks.com/
>>>
>>>
>>>
>>> On Wed, Jul 3, 2013 at 6:21 AM, Bing Jiang <[EMAIL PROTECTED]>wrote:
>>>
>>>> hi,all.
>>>> Using hadoop-2.0.5-alpha.
>>>> and I meet a problem that I should configure different
>>>> dfs.namenode.shared.edits.dir for different NameSpace.
>>>> So could dfs.namenode.shared.edits.dir item can support multiple
>>>> nameservies to avoid retaining multiple conf/hdfs-site.xml in different
>>>> Namenode?
>>>>
>>>> Thanks~
>>>>
>>>>
>>>>
>>>> --
>>>> Bing Jiang
>>>> Tel:(86)134-2619-1361
>>>> weibo: http://weibo.com/jiangbinglover
>>>> BLOG: http://blog.sina.com.cn/jiangbinglover
>>>> National Research Center for Intelligent Computing Systems
>>>> Institute of Computing technology
>>>> Graduate University of Chinese Academy of Science
>>>>
>>>
>>>
>>
>>
>> --
>> Bing Jiang
>> Tel:(86)134-2619-1361
>> weibo: http://weibo.com/jiangbinglover
>> BLOG: http://blog.sina.com.cn/jiangbinglover
>> National Research Center for Intelligent Computing Systems
>> Institute of Computing technology
>> Graduate University of Chinese Academy of Science
>>
>
>
--
Bing Jiang
Tel:(86)134-2619-1361
weibo: http://weibo.com/jiangbinglover
BLOG: http://blog.sina.com.cn/jiangbinglover
National Research Center for Intelligent Computing Systems
Institute of Computing technology
Graduate University of Chinese Academy of Science