Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - 【dfs.namenode.shared.edits.dir can support different NameServices】


Copy link to this message
-
Re: 【dfs.namenode.shared.edits.dir can support different NameServices】
Azuryy Yu 2013-07-04, 07:07
Hi Bing,
HA is not confilct with HDFS federation.

for example, you have two name services: cluster1, cluster2,

then,

<property>
    <name>dfs.namenode.shared.edits.dir</name>
    <value>qjournal://n1.com:8485;n2.com:8485/cluster1</value>
</property>

<property>
    <name>dfs.namenode.shared.edits.dir</name>
    <value>qjournal://n1.com:8485;n2.com:8485/cluster2</value>
 </property>
On Thu, Jul 4, 2013 at 2:46 PM, Bing Jiang <[EMAIL PROTECTED]> wrote:

> hi, Chris.
> I have traced the source code, and I find this issue comes from
> sbin/start-dfs.sh,
>
> SHARED_EDITS_DIR=$($HADOOP_PREFIX/bin/hdfs getconf -confKey
> dfs.namenode.shared.edits.dir 2>&-)
>
> If I set the suffix on dfs.namenode.shared.edits.dir.[namespace id].[nn
> id], it will get null. So please take consideration about HA on multiple
> nameservice.
> So, please change the way of launching JournalNodes in  shell scripts
> (sbin/start-dfs.sh)
>
>
> Thanks.
>
>
> 2013/7/4 Chris Nauroth <[EMAIL PROTECTED]>
>
>> Hello Bing,
>>
>> I have not tested this configuration myself, but from reading the code it
>> does appear that dfs.namenode.shared.edits.dir supports appending the
>> nameservice ID to the key, so that you can specify different directories
>> for different federated namenodes in a single hdfs-site.xml.
>>
>>
>> https://github.com/apache/hadoop-common/blob/branch-2.0.5-alpha/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java#L147
>>
>> Hope this helps,
>>
>> Chris Nauroth
>> Hortonworks
>> http://hortonworks.com/
>>
>>
>>
>> On Wed, Jul 3, 2013 at 6:21 AM, Bing Jiang <[EMAIL PROTECTED]>wrote:
>>
>>> hi,all.
>>> Using hadoop-2.0.5-alpha.
>>> and I meet a problem that I should configure different
>>> dfs.namenode.shared.edits.dir for different NameSpace.
>>> So could dfs.namenode.shared.edits.dir item can support multiple
>>> nameservies to avoid retaining multiple conf/hdfs-site.xml in different
>>> Namenode?
>>>
>>> Thanks~
>>>
>>>
>>>
>>> --
>>> Bing Jiang
>>> Tel:(86)134-2619-1361
>>> weibo: http://weibo.com/jiangbinglover
>>> BLOG: http://blog.sina.com.cn/jiangbinglover
>>> National Research Center for Intelligent Computing Systems
>>> Institute of Computing technology
>>> Graduate University of Chinese Academy of Science
>>>
>>
>>
>
>
> --
> Bing Jiang
> Tel:(86)134-2619-1361
> weibo: http://weibo.com/jiangbinglover
> BLOG: http://blog.sina.com.cn/jiangbinglover
> National Research Center for Intelligent Computing Systems
> Institute of Computing technology
> Graduate University of Chinese Academy of Science
>