Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Master node Question

Copy link to this message
RE: Master node Question
Jonathan Gray 2011-02-07, 19:42
Unfortunately, this is not the case for the HDFS NameNode.  HDFS does not support out-of-the-box backup master nodes.  There are a few different techniques people use for NameNode high availability... you should be able to find some stuff with a Google search, though none are especially simple.

In general, people run the NameNode on a more fault tolerant node... Dual PSUs, RAID, etc... I'd recommend at least mirrored RAID on the master nodes and dual power supplies.


> -----Original Message-----
> From: Joseph Coleman [mailto:[EMAIL PROTECTED]]
> Sent: Monday, February 07, 2011 11:38 AM
> Subject: Re: Master node Question
> Thank you for the reply that is very helpful I assume this is also the case for
> the HDFS master as well. Just list your masters  MasterA, MasterB and
> MasterC in the masters config files and also have your 2 of the three in the
> slaves as well I take it then execute your start commands from one master
> only example MasterA.
> On 2/7/11 1:29 PM, "Jonathan Gray" <[EMAIL PROTECTED]> wrote:
> >There is only one active HBase master at any given time, but there can
> >be any number of backup masters.  The failover is automated and
> >coordinated via ZooKeeper.  Regionservers and clients use ZooKeeper to
> >determine who is the current active master.  You can run with as many as
> you want.
> >
> >On larger clusters, I usually recommend around 5 master nodes.  One
> >possible configuration is to have all 5 with ZK (running on their own
> >dedicated spindles), one for the NameNode, one for the
> >SecondaryNameNode, and the other three can be HMasters (one will be
> the
> >normal HMaster, the other backups).
> >
> >JG
> >
> >> -----Original Message-----
> >> From: Joseph Coleman [mailto:[EMAIL PROTECTED]]
> >> Sent: Monday, February 07, 2011 11:05 AM
> >> Subject: Master node Question
> >>
> >> I am in the process of building out a clustered environment that is
> >>going to  start very small but grow extremely fast. I am just trying
> >>to figure out the
> >> following:
> >>
> >> 1. How many master nodes can I have?
> >>
> >> 2. If the answers is only 1 what happens if the master fails or dies.
> >>Does a
> >> slave take over and if so is the slave just referenced in the master
> >>files?
> >>
> >> Thanks for your help I am just trying to understand this before I
> >>order my  equipment. The build out will be a couple master nodes for
> >>redundancy,
> >>10
> >> data nodes running HDFS AND HBASE and a 2 to 3 node zookeeper cluster.
> >> We anticipate that out cluster will grow to 80 nodes before years end
> >>which is  why I am thinking of the setup they way I am if anyone has
> >>feedback for a  different setup please let me know.  Also if there is
> >>a doc that covers that  would be beneficial.
> >>
> >> Thanks for everyones help I do appreciate it.
> >