Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper, mail # user - Dynamic reconfiguration


Copy link to this message
-
Re: Dynamic reconfiguration
Jared Cantwell 2012-07-28, 23:52
I really liked the idea of just starting the config with the new server and
the leader of the old config, but I want to make sure I understand your
latest statement.  You're concluding that if a server were to start up at a
very later time with a config of old+self then we'd be safe still.  I think
that is because the only way a quorum can be established in this case is if
it includes a majority of the old config members, but those servers should
all (at least) know of a newer configuration (if they're even still
around), and they would be refusing to start unless the leader was in that
new configuration.  So as long as a given configuration doesn't *ever* have
a majority of servers that weren't in the config at some point then we're
good.  This of course makes sense (and has always been a terrible terrible
thing to do), but that can easily be avoided by never doing a union of
old+new.

~Jared

On Sat, Jul 28, 2012 at 5:36 PM, Alexander Shraer <[EMAIL PROTECTED]> wrote:

> On second thought, your scenario would not be possible even if you
> initiate a server with the current config plus only itself.
> I agree that you should not do union of old and new configs.
> Alex
>
> Sent from mobile
>
> On Jul 28, 2012, at 3:26 PM, Jared Cantwell <[EMAIL PROTECTED]>
> wrote:
>
> Having to "make up" a configuration for new servers that are non voting
> followers is something I keep getting stuck on, because if a couple of
> these servers start with just the wrong configuration they can all have an
> unspecified version and actually form a standalone quorum and not know a
> newer one exists. I would feel much better if new servers that are to
> become non voting followers could simply copy the configuration file from
> an existing voting participant (including  the version) and still start up.
> This is particularly an issue for us because servers with arbitrarily old
> configurations can start up at any time, and if the wrong things happen
> then they could form a quorum if they have the right "bootstrap"
> configurations with unspecified versions.
>
> Does this make sense as a concern? We may patch the server to allow
> starting up without its myid in the configuration.
>
> Jared
>
> On Jul 28, 2012, at 12:15 PM, Alexander Shraer <[EMAIL PROTECTED]> wrote:
>
> yes, if the server reboots (this is when it would read the config file).
> Otherwise, it has the last config in memory (this is held in a
> QuorumVerifier object in QuorumPeer) and it doesn't look in the config
> file.
>
> BTW the config file (when overwritten by the system) has an auto-generated
> version using which we know which config is later than which.  Users are
> not supposed to specify this version at all - its supposed to be managed by
> the system. If you replace the file and set the version to something low or
> not specify it at all, chances are that the config file will be overwritten
> during synchronization with the leader or during communication with other
> servers in FastLeaderElection.
> If you set it to something high, its possible that your server will be
> able to convince others that this is the latest config :)
>
> Alex
>
>
> On Sat, Jul 28, 2012 at 10:57 AM, Jared Cantwell <[EMAIL PROTECTED]
> > wrote:
>
>> No that you would want to do this, but simply overwriting a config file
>> would "uncommit" a configuration and make that server think the last
>> committed configuration was whatever is in the file?
>>
>
>