Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - zookeeper quorum verification


+
Rita 2011-11-30, 23:49
+
Jean-Daniel Cryans 2011-12-01, 00:14
+
Suraj Varma 2011-12-03, 14:15
+
Dave Barr 2011-12-03, 18:40
+
Suraj Varma 2011-12-04, 00:45
+
Shrijeet Paliwal 2011-12-04, 01:06
Copy link to this message
-
Re: zookeeper quorum verification
Shrijeet Paliwal 2011-12-04, 01:09
>>Now any HBase client that needs to connect to zookeeper, can talk to
quorum via "zookeeper-quorum.example.com"

Must add the reason above works is zookeeper uses  InetAddress.getAllByName
<http://docs.oracle.com/javase/1.3/docs/api/java/net/InetAddress.html#getAllByName(java.lang.String)>to
build the server address list

On Sat, Dec 3, 2011 at 5:06 PM, Shrijeet Paliwal <[EMAIL PROTECTED]>wrote:

> Suraj,
> Combine what J-D & David said. For example in our case
>
>    1. We let HBase manage zookeeper
>    2. Set the config in hbase-site.xml the way David mentioned
>    : <value>0.zookeeper,1.zookeeper,2.zookeeper,3.zookeeper,4.zookeeper</value>
>    3. Also have a DNS entry "zookeeper-quorum.example.com" with multiple
>    IPs (all mentioned above) resolving to this entry
>    4. Now any HBase client that needs to connect to zookeeper, can talk
>    to quorum via "zookeeper-quorum.example.com"
>
> One might think why we cant have hbase-site use ""
> zookeeper-quorum.example.com" as well. My understanding is when you start
> a zookeeper daemon on a server, the hbase managed zookeeper service will
> fetch the the default host name using hadoop's DNS.getDefaultHost method.
> The hostname returned may or may not find a match in the server list you
> mentioned in hbase-site.xml. If does not match you get the error mentioned
> here http://wiki.apache.org/hadoop/Hbase/Troubleshooting#A9.
>
> For example, in our case zookeeper host has two hostnames associated with
> default ethernet interface. First name is host.domain.com and second is
> the special DNS entry I talked about in (4). The DNS.getDefaultHost returns
> host.domain.com and zookeeper fails to start.
>
> I would be interested if some one has a different way of handling the
> situation I described.
>
> On Sat, Dec 3, 2011 at 4:45 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:
>
>> Yes - this makes sense. But, I thought what Rita suggested was a
>> single appquorum dns entry ... which was surprising.
>>
>> Hence my question.
>> --Suraj
>>
>> On Sat, Dec 3, 2011 at 10:40 AM, Dave Barr <[EMAIL PROTECTED]> wrote:
>> > What we do is make N.zookeeper.$DC.$DOMAIN entries in DNS.  We have
>> > one ZK cluster per DC.  Our configs then just point to
>> >
>> >
>> <value>0.zookeeper,1.zookeeper,2.zookeeper,3.zookeeper,4.zookeeper</value>
>> >
>> > --Dave
>> >
>> > On Sat, Dec 3, 2011 at 6:15 AM, Suraj Varma <[EMAIL PROTECTED]>
>> wrote:
>> >> J-D:
>> >> Did you mean that a _single_ dns entry returns all five ips belonging
>> >> to individual zk nodes?
>> >>
>> >> Is this used only by "clients" ... or even within the cluster?
>> >> And ... the zk nodes self-identify by IP ... and is this how region
>> >> server nodes reach out specifically to the "leader" zk node?
>> >> --Suraj
>> >>
>> >>
>> >> On Wed, Nov 30, 2011 at 4:14 PM, Jean-Daniel Cryans <
>> [EMAIL PROTECTED]> wrote:
>> >>> It's pretty much what we do, works well.
>> >>>
>> >>> J-D
>> >>>
>> >>> On Wed, Nov 30, 2011 at 3:49 PM, Rita <[EMAIL PROTECTED]> wrote:
>> >>>> Hello,
>> >>>>
>> >>>>
>> >>>> Previously, I assigned 5 servers as part of the zookeeper quorum.
>> >>>> Everything works fine but I was hard coding these 5 servers
>> everywhere and
>> >>>> I was thinking of creating a dns entry called appquorum which will
>> always
>> >>>> return these 5 servers IPs.
>> >>>>
>> >>>> Any thoughts about this?
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> --- Get your facts first, then you can distort them as you please.--
>>
>
>
+
Rita 2011-12-04, 15:03
+
Rita 2011-12-04, 15:09
+
Suraj Varma 2011-12-04, 17:18
+
Suraj Varma 2011-12-04, 17:36