Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> Growing a cluster


+
Jared Cantwell 2011-05-05, 02:16
+
Alexander Shraer 2011-05-05, 04:02
+
Jared Cantwell 2011-05-05, 12:57
+
Ted Dunning 2011-05-05, 14:30
+
Chang Song 2011-05-05, 14:31
+
Jared Cantwell 2011-05-05, 16:11
+
Patrick Hunt 2011-05-05, 16:28
Copy link to this message
-
Re: Growing a cluster
Thats Patrick, this is roughly the procedure we have are testing.

In your procedure, can (4) and (5) be performed simultaneously?

Also, am I correct in thinking that going from 1 to 3 server could be
dangerous for this reason:
1. start server 1 in standalone mode and write some data to it
2. follow you procedure with 3 servers
3. say during leader election, servers 2 and 3 form the quorum first and
elect server 2 as the leader.
4. server 2 believes the last txn to be 0, since it is a fresh cluster
5. now server 1 joins the quorum and knows about txn 100, but the other 2
servers do not, uh oh

Is this a real problem, or is there something preventing this case?  It
seems like you can only grow a cluster such that at least on member of the
old quorum MUST be present in the new quorum.  Is this correct?

~Jared

2011/5/5 Patrick Hunt <[EMAIL PROTECTED]>

> Hi Jared, you can go from 1 to 2+ nodes just fine:
>
> 1) change the configuration of server 1 from standalone to quorum
> based (be sure to list the new members of the ensemble)
> 2) add a myid file for server 1
> 3) setup new servers 2+
> 4) restart server 1
> 5) start servers 2+
>
> At this point the quorum should just come up. I just tried this
> example and it worked fine for me going from 1 to 3 (the data I
> created in standalone was available once I started the ensemble).
>
> Regards,
>
> Patrick
>
> 2011/5/5 Jared Cantwell <[EMAIL PROTECTED]>:
> > Great information guys-- this helps me understand what needs done when
> > expanding from 2 nodes to X nodes.
> >
> > Does anyone have insight on going from 1 node to 3 nodes?
> >
> > ~Jared
> >
> > 2011/5/5 Chang Song <[EMAIL PROTECTED]>
> >
> >>
> >> We are in a bit similar situation.
> >>
> >> 3 node -> 5 node ensemble.
> >>
> >> The only way to do this is the following.
> >>
> >> Assumption is that we have one DNS hostname for three zookeeper ensemble
> >> IP.
> >> Since five node ensemble allows 2 node failure for quorum, we can do
> >>
> >>
> >> 0. First all two new ensemble IPs
> >>
> >> 1. change all three existing node config (zoo.cfg) and add two new node
> >> information
> >>    Restart all three existing nodes in a sequence.
> >>
> >> 2. Replicate the new existing zoo.cfg to two new ensemble
> >>    Start Zookeeper on two new server
> >>
> >>
> >> You can do this in backward sequence (0 -> 2 -> 1)
> >> In your case, you can do 0 -> 2 -> 1, I think.
> >>
> >> Chang
> >>
> >>
> >>
> >> 2011. 5. 5., 오후 9:57, Jared Cantwell 작성:
> >>
> >> > It would be acceptable to me to do this non-dynamically and
> non-rolling
> >> as
> >> > well. For example, I can shut down all nodes, make necessary
> >> modifications
> >> > to config files, and then restart all nodes.  If I do this, should
> >> switching
> >> > from standalone mode to multi-node mode work?  Has anyone done this
> >> before?
> >> > Preliminary tests seem to work, but I haven't looked into all the race
> >> > conditions and such yet.
> >> >
> >> > ~Jared
> >> >
> >> > On Thu, May 5, 2011 at 12:02 AM, Alexander Shraer <
> [EMAIL PROTECTED]
> >> >wrote:
> >> >
> >> >> Hi Jared,
> >> >>
> >> >> Currently there is no support for adding and removing zookeeper nodes
> >> >> dynamically. See:
> >> >> https://issues.apache.org/jira/browse/ZOOKEEPER-107
> >> >>
> >> >> We're currently working to add this feature. However, AFAIK there is
> no
> >> >> intention to support
> >> >> transformation between standalone and multi-node modes, only
> membership
> >> >> changes in multi-node mode.
> >> >>
> >> >> Regards,
> >> >> Alex
> >> >>
> >> >>
> >> >>> -----Original Message-----
> >> >>> From: Jared Cantwell [mailto:[EMAIL PROTECTED]]
> >> >>> Sent: Wednesday, May 04, 2011 7:17 PM
> >> >>> To: [EMAIL PROTECTED]
> >> >>> Subject: Growing a cluster
> >> >>>
> >> >>> Hello,
> >> >>>
> >> >>> Say I was going to grow a cluster from 1 node to 3 nodes.  Is this
> >> >>> possible,
> >> >>> and what would be the recommended way?
> >> >>>
> >> >>> At first I was thinking that I could go from 1 to 2 and then 2 to 3,
+
Ted Dunning 2011-05-05, 17:07
+
Jared Cantwell 2011-05-05, 17:12
+
Ted Dunning 2011-05-05, 17:41
+
michael.boom 2013-11-05, 14:20
+
Camille Fournier 2013-11-05, 15:44
+
Alexander Shraer 2013-11-05, 16:06
+
Ted Dunning 2011-05-05, 17:05
+
Jeremy Stribling 2011-05-05, 16:29
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB