Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> zookeeper quorum verification


+
Rita 2011-11-30, 23:49
+
Jean-Daniel Cryans 2011-12-01, 00:14
+
Suraj Varma 2011-12-03, 14:15
+
Dave Barr 2011-12-03, 18:40
+
Suraj Varma 2011-12-04, 00:45
Copy link to this message
-
Re: zookeeper quorum verification
Suraj,
Combine what J-D & David said. For example in our case

   1. We let HBase manage zookeeper
   2. Set the config in hbase-site.xml the way David mentioned
   : <value>0.zookeeper,1.zookeeper,2.zookeeper,3.zookeeper,4.zookeeper</value>
   3. Also have a DNS entry "zookeeper-quorum.example.com" with multiple
   IPs (all mentioned above) resolving to this entry
   4. Now any HBase client that needs to connect to zookeeper, can talk to
   quorum via "zookeeper-quorum.example.com"

One might think why we cant have hbase-site use ""
zookeeper-quorum.example.com" as well. My understanding is when you start a
zookeeper daemon on a server, the hbase managed zookeeper service will
fetch the the default host name using hadoop's DNS.getDefaultHost method.
The hostname returned may or may not find a match in the server list you
mentioned in hbase-site.xml. If does not match you get the error mentioned
here http://wiki.apache.org/hadoop/Hbase/Troubleshooting#A9.

For example, in our case zookeeper host has two hostnames associated with
default ethernet interface. First name is host.domain.com and second is the
special DNS entry I talked about in (4). The DNS.getDefaultHost returns
host.domain.com and zookeeper fails to start.

I would be interested if some one has a different way of handling the
situation I described.

On Sat, Dec 3, 2011 at 4:45 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:

> Yes - this makes sense. But, I thought what Rita suggested was a
> single appquorum dns entry ... which was surprising.
>
> Hence my question.
> --Suraj
>
> On Sat, Dec 3, 2011 at 10:40 AM, Dave Barr <[EMAIL PROTECTED]> wrote:
> > What we do is make N.zookeeper.$DC.$DOMAIN entries in DNS.  We have
> > one ZK cluster per DC.  Our configs then just point to
> >
> >
> <value>0.zookeeper,1.zookeeper,2.zookeeper,3.zookeeper,4.zookeeper</value>
> >
> > --Dave
> >
> > On Sat, Dec 3, 2011 at 6:15 AM, Suraj Varma <[EMAIL PROTECTED]> wrote:
> >> J-D:
> >> Did you mean that a _single_ dns entry returns all five ips belonging
> >> to individual zk nodes?
> >>
> >> Is this used only by "clients" ... or even within the cluster?
> >> And ... the zk nodes self-identify by IP ... and is this how region
> >> server nodes reach out specifically to the "leader" zk node?
> >> --Suraj
> >>
> >>
> >> On Wed, Nov 30, 2011 at 4:14 PM, Jean-Daniel Cryans <
> [EMAIL PROTECTED]> wrote:
> >>> It's pretty much what we do, works well.
> >>>
> >>> J-D
> >>>
> >>> On Wed, Nov 30, 2011 at 3:49 PM, Rita <[EMAIL PROTECTED]> wrote:
> >>>> Hello,
> >>>>
> >>>>
> >>>> Previously, I assigned 5 servers as part of the zookeeper quorum.
> >>>> Everything works fine but I was hard coding these 5 servers
> everywhere and
> >>>> I was thinking of creating a dns entry called appquorum which will
> always
> >>>> return these 5 servers IPs.
> >>>>
> >>>> Any thoughts about this?
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> --- Get your facts first, then you can distort them as you please.--
>
+
Shrijeet Paliwal 2011-12-04, 01:09
+
Rita 2011-12-04, 15:03
+
Rita 2011-12-04, 15:09
+
Suraj Varma 2011-12-04, 17:18
+
Suraj Varma 2011-12-04, 17:36
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB