Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> Zookeeper multiserver setup?


+
Something Something 2012-05-08, 21:31
+
Jordan Zimmerman 2012-05-08, 21:36
+
Something Something 2012-05-08, 21:43
+
Jordan Zimmerman 2012-05-08, 21:45
+
Something Something 2012-05-08, 21:54
+
Adam Rosien 2012-05-09, 15:33
+
Patrick Hunt 2012-05-09, 22:24
+
Adam Rosien 2012-05-10, 15:16
+
Patrick Hunt 2012-05-17, 21:27
+
Jordan Zimmerman 2012-05-08, 22:05
+
Something Something 2012-05-08, 22:13
Copy link to this message
-
Re: Zookeeper multiserver setup?
For most uses, 3 instances is enough. For heavy usage, 5 instances can be
used. ZooKeeper write performance degrades with the number of instances
added. There's a lot of doc on the net about this.

A ZNode is a data entry in the ZooKeeper database. An instance is a server
running the ZooKeeper software.

-JZ

On 5/8/12 3:13 PM, "Something Something" <[EMAIL PROTECTED]> wrote:

>We have a cluster of 17 machines.  Somewhere in the HBase documentation I
>read - usually we should divide by 2 & use an odd number.  May be I
>mis-read.  That's why I started Zookeeper on 9 machines.  I guess, I don't
>understand the difference between Nodes & Instance.  I expected ZKNodes to
>be 9.  Please explain.  Thanks a lot for your help.
>
>On Tue, May 8, 2012 at 3:05 PM, Jordan Zimmerman
><[EMAIL PROTECTED]>wrote:
>
>> You have 9 instances? That's a lot. Why so many?
>>
>> I believe the "Node" field in the stat is the number of ZKNodes, not
>> instances in the ensemble. That stat looks good. It means that that
>> instance is serving requests as a "follower" instance (one of the
>> instances will be the leader).
>>
>> -JZ
>>
>> On 5/8/12 2:54 PM, "Something Something" <[EMAIL PROTECTED]>
>>wrote:
>>
>> >Hmm...
>> >
>> >
>> >Trying 127.0.0.1...
>> >Connected to localhost.
>> >Escape character is '^]'.
>> >stat
>> >Zookeeper version: 3.3.4-cdh3u3--1, built on 01/26/2012 20:09 GMT
>> >Clients:
>> > /127.0.0.1:42510[0](queued=0,recved=1,sent=0)
>> >
>> >Latency min/avg/max: 0/0/0
>> >Received: 4
>> >Sent: 3
>> >Outstanding: 0
>> >Zxid: 0x100000000
>> >Mode: follower
>> >Node count: 11
>> >Connection closed by foreign host.
>> >
>> >
>> >Not sure I understand this output.  I have 9 nodes in the ensemble.
>> >
>> >On Tue, May 8, 2012 at 2:45 PM, Jordan Zimmerman
>> ><[EMAIL PROTECTED]>wrote:
>> >
>> >> Yes - they should. Keep in mind that the zoo.cfg, and the ID file
>>must
>> >>be
>> >> correct and in agreement on each instance. You can do a stat on each
>> >> instance to see what state it's in:
>> >>
>> >>    >telnet localhost 2181
>> >>     stat
>> >>
>> >> -JZ
>> >>
>> >> On 5/8/12 2:43 PM, "Something Something" <[EMAIL PROTECTED]>
>> >>wrote:
>> >>
>> >> >Thanks Jordan.  Will definitely look at the cluster management tool.
>> >>In
>> >> >the mean time, am I correct with my assumption in #4?  Once all of
>>them
>> >> >are
>> >> >started they would automagically start talking to each other, right?
>> >> >
>> >> >
>> >> >On Tue, May 8, 2012 at 2:36 PM, Jordan Zimmerman
>> >> ><[EMAIL PROTECTED]>wrote:
>> >> >
>> >> >> Yes, you must start them all individually. ZooKeeper does not
>> >>currently
>> >> >> have any cluster-wide management tools built in.
>> >> >>
>> >> >> FYI - we've open sourced a cluster management tool for ZooKeeper.
>> >>Please
>> >> >> have a look:
>> >> >>
>> >> >>        https://github.com/Netflix/exhibitor
>> >> >>
>> >> >> On 5/8/12 2:31 PM, "Something Something"
>><[EMAIL PROTECTED]>
>> >> >>wrote:
>> >> >>
>> >> >> >Hello,
>> >> >> >
>> >> >> >Trying to follow directions from here:
>> >> >> >
>> >> >>
>> >> >>
>> >>
>> >>
>>
>>http://zookeeper.apache.org/doc/trunk/zookeeperAdmin.html#sc_zkMulitServe
>> >> >>r
>> >> >> >Setup
>> >> >> >
>> >> >> >to setup a cluster of zookeepers.
>> >> >> >
>> >> >> >Questions:
>> >> >> >
>> >> >> >1)  If everything is setup correctly, do I have to go to each
>> >>machine
>> >> >>and
>> >> >> >start Zookeeper server in each of them individually?
>> >> >> >2)  I was thinking if I start it on one master machine (well
>> >>there's no
>> >> >> >real master) then all zookeepers in the quorum would be started,
>>but
>> >> >>this
>> >> >> >is not how it works correct?
>> >> >> >3)  If I start individually, then I get messages saying the other
>> >> >> >zookeepers haven't started.  Is that just a warning?
>> >> >> >4)  When all of them are started, would they all be able to talk
>>to
>> >> >>each
>> >> >> >other?
>> >> >> >
>> >> >> >Sorry for the dumb questions and thanks for the help.
+
Something Something 2012-05-08, 22:46
+
Patrick Hunt 2012-05-09, 22:21