Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> Zookeeper Configuration Sync


Copy link to this message
-
Re: Zookeeper Configuration Sync
To your question, you can initialize the joiners config with a list
including the current servers and the new one. Or current leader and new
server - works too but more fragile. If two servers join they shouldn't
have each other in their initial configs. See ZK-1660 for user manual.
On Jul 8, 2013 1:27 PM, "Mohammad Shamma" <[EMAIL PROTECTED]> wrote:

> Sorry for the late reply,
>
>
> On Fri, Jun 21, 2013 at 11:47 PM, Alexander Shraer <[EMAIL PROTECTED]
> >wrote:
>
> > Hi Mohammad,
> >
> > +1 for the unique ensemble identifier request. We actually discussed
> > this a long time ago but somehow never got to do this.
> > Can you open a JIRA for this ?
> >
>
> I will do that.
>
>
> >
> > Suppose that server A has the latest log but only talks with server B
> > during leader election (C is down or slow). A doesn't know whether
> > or not the latest operations in its log are committed (in this case C
> > would have them, but A doesn't know if this is the case). So to be
> > safe
> > everything in A's log gets committed in this case.
>
>
> > We took the approach that a reconfiguration is treated similarly to
> > normal data updates. When a server has the most up-to-date log and
> > talks with a majority during leader election, it will be elected
> > leader and commit its log to the other servers. It won't truncate its
> > log even
> > if its clear that some operations were not committed. This is true for
> > normal updates as well as for reconfigurations.
> >
> > BTW, I'm not sure why you are shutting down servers or clearing the
> > data during reconfigurations, or why you're manually changing config
> > files.
>
>
> The reason I am shutting down and clearing the data of the "to be added"
> servers is to delete their history as the intention here is to make them
> join a new fresh deployment.
>
> You can add servers to the ensemble by invoking the reconfig command
> > and this will make all the necessary changes to the config files,
> > including specifying the right config version.
> >
>
> If that is the case, what goes into the zookeeper config file of a new
> zookeeper server that is supposed to join an existing ensemble?
>
>
> >
> > Alex
> >
> >
> > On Fri, Jun 21, 2013 at 3:00 PM, Mohammad Shamma
> > <[EMAIL PROTECTED]> wrote:
> > > I have a use case where I dynamically grow a zookeeper ensemble on the
> > same
> > > fixed set of machines multiple times. In each iteration, the ensemble
> is
> > > grown incrementally till it consists of "n" servers. I will refer to
> the
> > > machines hosting the servers as zk-1, zk-2, ..., zk-n.
> > >
> > > At the beginning of each iteration, I wipe out the zookeeper data
> > > directories of zk-1 and zk-2, then statically configure the zookeeper
> > > servers on both of them to form a 2-way ensemble. After that, I start
> > > growing the ensemble incrementally by reconfiguring the zookeeper
> > ensemble
> > > to include zk-i, then clearing, configure and starting the zookeeper
> > server
> > > on zk-i (that is for i in range(2,n)).
> > >
> > > I was not shutting down or cleaning up the previous ensemble zookeeper
> > > servers at the end of each iteration. After initializing the 2-way
> > ensemble
> > > on zk-1 and zk-2, I observed that the servers from the old deployment
> > were
> > > contacting the servers of the new ensemble and triggering an ensemble
> > > reconfiguration. A quick look at the code seems to suggest that this is
> > > simply triggered by the virtue that the config version value of the old
> > > deployment server is higher than that of that found on the new ensemble
> > > servers. Can anyone confirm my understanding of this behaviour of
> > zookeeper?
> > >
> > > I also noticed that his reconfiguration holds true for n=3. For example
> > > lets say zookeeper  servers on zk-1 and zk-2 are freshly configured to
> > form
> > > a 2-way ensemble, and zk-3 contains a leftover server that was part of
> an
> > > older 3-way ensemble (that included two obselete servers on zk-1 and
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB