Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> Zookeeper multiserver setup?


+
Something Something 2012-05-08, 21:31
+
Jordan Zimmerman 2012-05-08, 21:36
+
Something Something 2012-05-08, 21:43
+
Jordan Zimmerman 2012-05-08, 21:45
+
Something Something 2012-05-08, 21:54
+
Adam Rosien 2012-05-09, 15:33
+
Patrick Hunt 2012-05-09, 22:24
+
Adam Rosien 2012-05-10, 15:16
+
Patrick Hunt 2012-05-17, 21:27
+
Jordan Zimmerman 2012-05-08, 22:05
+
Something Something 2012-05-08, 22:13
Copy link to this message
-
Re: Zookeeper multiserver setup?
For most uses, 3 instances is enough. For heavy usage, 5 instances can be
used. ZooKeeper write performance degrades with the number of instances
added. There's a lot of doc on the net about this.

A ZNode is a data entry in the ZooKeeper database. An instance is a server
running the ZooKeeper software.

-JZ

On 5/8/12 3:13 PM, "Something Something" <[EMAIL PROTECTED]> wrote:

>We have a cluster of 17 machines.  Somewhere in the HBase documentation I
>read - usually we should divide by 2 & use an odd number.  May be I
>mis-read.  That's why I started Zookeeper on 9 machines.  I guess, I don't
>understand the difference between Nodes & Instance.  I expected ZKNodes to
>be 9.  Please explain.  Thanks a lot for your help.
>
>On Tue, May 8, 2012 at 3:05 PM, Jordan Zimmerman
><[EMAIL PROTECTED]>wrote:
>
>> You have 9 instances? That's a lot. Why so many?
>>
>> I believe the "Node" field in the stat is the number of ZKNodes, not
>> instances in the ensemble. That stat looks good. It means that that
>> instance is serving requests as a "follower" instance (one of the
>> instances will be the leader).
>>
>> -JZ
>>
>> On 5/8/12 2:54 PM, "Something Something" <[EMAIL PROTECTED]>
>>wrote:
>>
>> >Hmm...
>> >
>> >
>> >Trying 127.0.0.1...
>> >Connected to localhost.
>> >Escape character is '^]'.
>> >stat
>> >Zookeeper version: 3.3.4-cdh3u3--1, built on 01/26/2012 20:09 GMT
>> >Clients:
>> > /127.0.0.1:42510[0](queued=0,recved=1,sent=0)
>> >
>> >Latency min/avg/max: 0/0/0
>> >Received: 4
>> >Sent: 3
>> >Outstanding: 0
>> >Zxid: 0x100000000
>> >Mode: follower
>> >Node count: 11
>> >Connection closed by foreign host.
>> >
>> >
>> >Not sure I understand this output.  I have 9 nodes in the ensemble.
>> >
>> >On Tue, May 8, 2012 at 2:45 PM, Jordan Zimmerman
>> ><[EMAIL PROTECTED]>wrote:
>> >
>> >> Yes - they should. Keep in mind that the zoo.cfg, and the ID file
>>must
>> >>be
>> >> correct and in agreement on each instance. You can do a stat on each
>> >> instance to see what state it's in:
>> >>
>> >>    >telnet localhost 2181
>> >>     stat
>> >>
>> >> -JZ
>> >>
>> >> On 5/8/12 2:43 PM, "Something Something" <[EMAIL PROTECTED]>
>> >>wrote:
>> >>
>> >> >Thanks Jordan.  Will definitely look at the cluster management tool.
>> >>In
>> >> >the mean time, am I correct with my assumption in #4?  Once all of
>>them
>> >> >are
>> >> >started they would automagically start talking to each other, right?
>> >> >
>> >> >
>> >> >On Tue, May 8, 2012 at 2:36 PM, Jordan Zimmerman
>> >> ><[EMAIL PROTECTED]>wrote:
>> >> >
>> >> >> Yes, you must start them all individually. ZooKeeper does not
>> >>currently
>> >> >> have any cluster-wide management tools built in.
>> >> >>
>> >> >> FYI - we've open sourced a cluster management tool for ZooKeeper.
>> >>Please
>> >> >> have a look:
>> >> >>
>> >> >>        https://github.com/Netflix/exhibitor
>> >> >>
>> >> >> On 5/8/12 2:31 PM, "Something Something"
>><[EMAIL PROTECTED]>
>> >> >>wrote:
>> >> >>
>> >> >> >Hello,
>> >> >> >
>> >> >> >Trying to follow directions from here:
>> >> >> >
>> >> >>
>> >> >>
>> >>
>> >>
>>
>>http://zookeeper.apache.org/doc/trunk/zookeeperAdmin.html#sc_zkMulitServe
>> >> >>r
>> >> >> >Setup
>> >> >> >
>> >> >> >to setup a cluster of zookeepers.
>> >> >> >
>> >> >> >Questions:
>> >> >> >
>> >> >> >1)  If everything is setup correctly, do I have to go to each
>> >>machine
>> >> >>and
>> >> >> >start Zookeeper server in each of them individually?
>> >> >> >2)  I was thinking if I start it on one master machine (well
>> >>there's no
>> >> >> >real master) then all zookeepers in the quorum would be started,
>>but
>> >> >>this
>> >> >> >is not how it works correct?
>> >> >> >3)  If I start individually, then I get messages saying the other
>> >> >> >zookeepers haven't started.  Is that just a warning?
>> >> >> >4)  When all of them are started, would they all be able to talk
>>to
>> >> >>each
>> >> >> >other?
>> >> >> >
>> >> >> >Sorry for the dumb questions and thanks for the help.
+
Something Something 2012-05-08, 22:46
+
Patrick Hunt 2012-05-09, 22:21
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB