Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> Zookeeper Configuration Sync


Copy link to this message
-
Re: Zookeeper Configuration Sync
That's not necessary - during the reconfiguration the config files are
updated automatically.
On Jul 8, 2013 3:08 PM, "Mohammad Shamma" <[EMAIL PROTECTED]> wrote:

> Thanks for the documentation draft reference.
>
> I have been using the configuration information read using the zk client
> (post making the reconfig call) as a basis for generating the config file
> of the new zookeeper server.
>
>
> On Mon, Jul 8, 2013 at 2:48 PM, Alexander Shraer <[EMAIL PROTECTED]>
> wrote:
>
> > To your question, you can initialize the joiners config with a list
> > including the current servers and the new one. Or current leader and new
> > server - works too but more fragile. If two servers join they shouldn't
> > have each other in their initial configs. See ZK-1660 for user manual.
> > On Jul 8, 2013 1:27 PM, "Mohammad Shamma" <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Sorry for the late reply,
> > >
> > >
> > > On Fri, Jun 21, 2013 at 11:47 PM, Alexander Shraer <[EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > Hi Mohammad,
> > > >
> > > > +1 for the unique ensemble identifier request. We actually discussed
> > > > this a long time ago but somehow never got to do this.
> > > > Can you open a JIRA for this ?
> > > >
> > >
> > > I will do that.
> > >
> > >
> > > >
> > > > Suppose that server A has the latest log but only talks with server B
> > > > during leader election (C is down or slow). A doesn't know whether
> > > > or not the latest operations in its log are committed (in this case C
> > > > would have them, but A doesn't know if this is the case). So to be
> > > > safe
> > > > everything in A's log gets committed in this case.
> > >
> > >
> > > > We took the approach that a reconfiguration is treated similarly to
> > > > normal data updates. When a server has the most up-to-date log and
> > > > talks with a majority during leader election, it will be elected
> > > > leader and commit its log to the other servers. It won't truncate its
> > > > log even
> > > > if its clear that some operations were not committed. This is true
> for
> > > > normal updates as well as for reconfigurations.
> > > >
> > > > BTW, I'm not sure why you are shutting down servers or clearing the
> > > > data during reconfigurations, or why you're manually changing config
> > > > files.
> > >
> > >
> > > The reason I am shutting down and clearing the data of the "to be
> added"
> > > servers is to delete their history as the intention here is to make
> them
> > > join a new fresh deployment.
> > >
> > > You can add servers to the ensemble by invoking the reconfig command
> > > > and this will make all the necessary changes to the config files,
> > > > including specifying the right config version.
> > > >
> > >
> > > If that is the case, what goes into the zookeeper config file of a new
> > > zookeeper server that is supposed to join an existing ensemble?
> > >
> > >
> > > >
> > > > Alex
> > > >
> > > >
> > > > On Fri, Jun 21, 2013 at 3:00 PM, Mohammad Shamma
> > > > <[EMAIL PROTECTED]> wrote:
> > > > > I have a use case where I dynamically grow a zookeeper ensemble on
> > the
> > > > same
> > > > > fixed set of machines multiple times. In each iteration, the
> ensemble
> > > is
> > > > > grown incrementally till it consists of "n" servers. I will refer
> to
> > > the
> > > > > machines hosting the servers as zk-1, zk-2, ..., zk-n.
> > > > >
> > > > > At the beginning of each iteration, I wipe out the zookeeper data
> > > > > directories of zk-1 and zk-2, then statically configure the
> zookeeper
> > > > > servers on both of them to form a 2-way ensemble. After that, I
> start
> > > > > growing the ensemble incrementally by reconfiguring the zookeeper
> > > > ensemble
> > > > > to include zk-i, then clearing, configure and starting the
> zookeeper
> > > > server
> > > > > on zk-i (that is for i in range(2,n)).
> > > > >
> > > > > I was not shutting down or cleaning up the previous ensemble
> > zookeeper
> > > > > servers at the end of each iteration. After initializing the 2-way