|
|
-
HDFS HA with HDFS Federation
lohit 2012-10-31, 20:08
Hi Devs,
I am trying to test HDFS Federation and HDFS HA on same cluster. My setup has 2 federated NameNodes out of which only one NameNode is in HA mode. It looks like all configs has good structure to distinguish between the namenode. For example dfs.namenode.rpc-address.cluster.namenode is kind of unique. This enables me to have same hdfs-site.xml across all nodes on cluster. But some configs like dfs.namenode.shared.edits.dir does not have this property. This forces me to have different hdfs-site.xml on different nodes. Has anyone run into this situation? Any suggestions to overcome this?
-- Have a Nice Day! Lohit
+
lohit 2012-10-31, 20:08
-
Re: HDFS HA with HDFS Federation
Todd Lipcon 2012-10-31, 20:10
Hi Lohit,
The shared edits dir config is also optionally per-namenode -- i.e you can specify dfs.namenode.shared.edits.dir.namespace1 or even dfs.namenode.shared.edits.dir.namespace1.nn2. For reference in the code, check out NameNode.NAMENODE_SPECIFIC_KEYS.
-Todd
On Wed, Oct 31, 2012 at 1:08 PM, lohit <[EMAIL PROTECTED]> wrote:
> Hi Devs, > > I am trying to test HDFS Federation and HDFS HA on same cluster. > My setup has 2 federated NameNodes out of which only one NameNode is in HA > mode. > It looks like all configs has good structure to distinguish between the > namenode. > For example dfs.namenode.rpc-address.cluster.namenode is kind of unique. > This enables me to have same hdfs-site.xml across all nodes on cluster. > But some configs like dfs.namenode.shared.edits.dir does not have this > property. > This forces me to have different hdfs-site.xml on different nodes. > Has anyone run into this situation? Any suggestions to overcome this? > > -- > Have a Nice Day! > Lohit >
-- Todd Lipcon Software Engineer, Cloudera
+
Todd Lipcon 2012-10-31, 20:10
-
Re: HDFS HA with HDFS Federation
lohit 2012-10-31, 20:15
This is good. Thanks
2012/10/31 Todd Lipcon <[EMAIL PROTECTED]>
> Hi Lohit, > > The shared edits dir config is also optionally per-namenode -- i.e you can > specify dfs.namenode.shared.edits.dir.namespace1 or > even dfs.namenode.shared.edits.dir.namespace1.nn2. For reference in the > code, check out NameNode.NAMENODE_SPECIFIC_KEYS. > > -Todd > > On Wed, Oct 31, 2012 at 1:08 PM, lohit <[EMAIL PROTECTED]> wrote: > > > Hi Devs, > > > > I am trying to test HDFS Federation and HDFS HA on same cluster. > > My setup has 2 federated NameNodes out of which only one NameNode is in > HA > > mode. > > It looks like all configs has good structure to distinguish between the > > namenode. > > For example dfs.namenode.rpc-address.cluster.namenode is kind of unique. > > This enables me to have same hdfs-site.xml across all nodes on cluster. > > But some configs like dfs.namenode.shared.edits.dir does not have this > > property. > > This forces me to have different hdfs-site.xml on different nodes. > > Has anyone run into this situation? Any suggestions to overcome this? > > > > -- > > Have a Nice Day! > > Lohit > > > > > > -- > Todd Lipcon > Software Engineer, Cloudera >
-- Have a Nice Day! Lohit
+
lohit 2012-10-31, 20:15
|
|
All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by
Sematext