Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: 【dfs.namenode.shared.edits.dir can support different NameServices】


+
Chris Nauroth 2013-07-03, 18:02
Copy link to this message
-
Re: 【dfs.namenode.shared.edits.dir can support different NameServices】
hi, Chris.
I have traced the source code, and I find this issue comes from
sbin/start-dfs.sh,

SHARED_EDITS_DIR=$($HADOOP_PREFIX/bin/hdfs getconf -confKey
dfs.namenode.shared.edits.dir 2>&-)

If I set the suffix on dfs.namenode.shared.edits.dir.[namespace id].[nn
id], it will get null. So please take consideration about HA on multiple
nameservice.
So, please change the way of launching JournalNodes in  shell scripts
(sbin/start-dfs.sh)
Thanks.
2013/7/4 Chris Nauroth <[EMAIL PROTECTED]>

> Hello Bing,
>
> I have not tested this configuration myself, but from reading the code it
> does appear that dfs.namenode.shared.edits.dir supports appending the
> nameservice ID to the key, so that you can specify different directories
> for different federated namenodes in a single hdfs-site.xml.
>
>
> https://github.com/apache/hadoop-common/blob/branch-2.0.5-alpha/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java#L147
>
> Hope this helps,
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Wed, Jul 3, 2013 at 6:21 AM, Bing Jiang <[EMAIL PROTECTED]>wrote:
>
>> hi,all.
>> Using hadoop-2.0.5-alpha.
>> and I meet a problem that I should configure different
>> dfs.namenode.shared.edits.dir for different NameSpace.
>> So could dfs.namenode.shared.edits.dir item can support multiple
>> nameservies to avoid retaining multiple conf/hdfs-site.xml in different
>> Namenode?
>>
>> Thanks~
>>
>>
>>
>> --
>> Bing Jiang
>> Tel:(86)134-2619-1361
>> weibo: http://weibo.com/jiangbinglover
>> BLOG: http://blog.sina.com.cn/jiangbinglover
>> National Research Center for Intelligent Computing Systems
>> Institute of Computing technology
>> Graduate University of Chinese Academy of Science
>>
>
>
--
Bing Jiang
Tel:(86)134-2619-1361
weibo: http://weibo.com/jiangbinglover
BLOG: http://blog.sina.com.cn/jiangbinglover
National Research Center for Intelligent Computing Systems
Institute of Computing technology
Graduate University of Chinese Academy of Science
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB