Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - zookeeper quorum verification


Copy link to this message
-
Re: zookeeper quorum verification
Shrijeet Paliwal 2011-12-04, 01:06
Suraj,
Combine what J-D & David said. For example in our case

   1. We let HBase manage zookeeper
   2. Set the config in hbase-site.xml the way David mentioned
   : <value>0.zookeeper,1.zookeeper,2.zookeeper,3.zookeeper,4.zookeeper</value>
   3. Also have a DNS entry "zookeeper-quorum.example.com" with multiple
   IPs (all mentioned above) resolving to this entry
   4. Now any HBase client that needs to connect to zookeeper, can talk to
   quorum via "zookeeper-quorum.example.com"

One might think why we cant have hbase-site use ""
zookeeper-quorum.example.com" as well. My understanding is when you start a
zookeeper daemon on a server, the hbase managed zookeeper service will
fetch the the default host name using hadoop's DNS.getDefaultHost method.
The hostname returned may or may not find a match in the server list you
mentioned in hbase-site.xml. If does not match you get the error mentioned
here http://wiki.apache.org/hadoop/Hbase/Troubleshooting#A9.

For example, in our case zookeeper host has two hostnames associated with
default ethernet interface. First name is host.domain.com and second is the
special DNS entry I talked about in (4). The DNS.getDefaultHost returns
host.domain.com and zookeeper fails to start.

I would be interested if some one has a different way of handling the
situation I described.

On Sat, Dec 3, 2011 at 4:45 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:

> Yes - this makes sense. But, I thought what Rita suggested was a
> single appquorum dns entry ... which was surprising.
>
> Hence my question.
> --Suraj
>
> On Sat, Dec 3, 2011 at 10:40 AM, Dave Barr <[EMAIL PROTECTED]> wrote:
> > What we do is make N.zookeeper.$DC.$DOMAIN entries in DNS.  We have
> > one ZK cluster per DC.  Our configs then just point to
> >
> >
> <value>0.zookeeper,1.zookeeper,2.zookeeper,3.zookeeper,4.zookeeper</value>
> >
> > --Dave
> >
> > On Sat, Dec 3, 2011 at 6:15 AM, Suraj Varma <[EMAIL PROTECTED]> wrote:
> >> J-D:
> >> Did you mean that a _single_ dns entry returns all five ips belonging
> >> to individual zk nodes?
> >>
> >> Is this used only by "clients" ... or even within the cluster?
> >> And ... the zk nodes self-identify by IP ... and is this how region
> >> server nodes reach out specifically to the "leader" zk node?
> >> --Suraj
> >>
> >>
> >> On Wed, Nov 30, 2011 at 4:14 PM, Jean-Daniel Cryans <
> [EMAIL PROTECTED]> wrote:
> >>> It's pretty much what we do, works well.
> >>>
> >>> J-D
> >>>
> >>> On Wed, Nov 30, 2011 at 3:49 PM, Rita <[EMAIL PROTECTED]> wrote:
> >>>> Hello,
> >>>>
> >>>>
> >>>> Previously, I assigned 5 servers as part of the zookeeper quorum.
> >>>> Everything works fine but I was hard coding these 5 servers
> everywhere and
> >>>> I was thinking of creating a dns entry called appquorum which will
> always
> >>>> return these 5 servers IPs.
> >>>>
> >>>> Any thoughts about this?
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> --- Get your facts first, then you can distort them as you please.--
>