Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper, mail # user - Dynamic reconfiguration

Copy link to this message
Re: Dynamic reconfiguration
Alexander Shraer 2012-07-29, 01:00
Yes, this way two new guys will not connect to each other because they dont know about each other. Every quorum including a bew guy must include at least one old guy that knows the latest config and state. For example if the current config is ABCDE, its possible that D and never heard any ops. If we're adding F then FDE is not a majority of ABCDEF there must be at least one that knows all commited state. This is true only for majority quorums (which is the one being used by probably everyone). For hierarchical quorums I'd use the meathod of leader+self.


Sent from mobile

On Jul 28, 2012, at 4:52 PM, Jared Cantwell <[EMAIL PROTECTED]> wrote:

> I really liked the idea of just starting the config with the new server and the leader of the old config, but I want to make sure I understand your latest statement.  You're concluding that if a server were to start up at a very later time with a config of old+self then we'd be safe still.  I think that is because the only way a quorum can be established in this case is if it includes a majority of the old config members, but those servers should all (at least) know of a newer configuration (if they're even still around), and they would be refusing to start unless the leader was in that new configuration.  So as long as a given configuration doesn't ever have a majority of servers that weren't in the config at some point then we're good.  This of course makes sense (and has always been a terrible terrible thing to do), but that can easily be avoided by never doing a union of old+new.
> ~Jared
> On Sat, Jul 28, 2012 at 5:36 PM, Alexander Shraer <[EMAIL PROTECTED]> wrote:
> On second thought, your scenario would not be possible even if you initiate a server with the current config plus only itself.
> I agree that you should not do union of old and new configs.
> Alex
> Sent from mobile
> On Jul 28, 2012, at 3:26 PM, Jared Cantwell <[EMAIL PROTECTED]> wrote:
>> Having to "make up" a configuration for new servers that are non voting followers is something I keep getting stuck on, because if a couple of these servers start with just the wrong configuration they can all have an unspecified version and actually form a standalone quorum and not know a newer one exists. I would feel much better if new servers that are to become non voting followers could simply copy the configuration file from an existing voting participant (including  the version) and still start up. This is particularly an issue for us because servers with arbitrarily old configurations can start up at any time, and if the wrong things happen then they could form a quorum if they have the right "bootstrap" configurations with unspecified versions.
>> Does this make sense as a concern? We may patch the server to allow starting up without its myid in the configuration.
>> Jared
>> On Jul 28, 2012, at 12:15 PM, Alexander Shraer <[EMAIL PROTECTED]> wrote:
>>> yes, if the server reboots (this is when it would read the config file).
>>> Otherwise, it has the last config in memory (this is held in a QuorumVerifier object in QuorumPeer) and it doesn't look in the config file.
>>> BTW the config file (when overwritten by the system) has an auto-generated version using which we know which config is later than which.  Users are not supposed to specify this version at all - its supposed to be managed by the system. If you replace the file and set the version to something low or not specify it at all, chances are that the config file will be overwritten during synchronization with the leader or during communication with other servers in FastLeaderElection.
>>> If you set it to something high, its possible that your server will be able to convince others that this is the latest config :)
>>> Alex
>>> On Sat, Jul 28, 2012 at 10:57 AM, Jared Cantwell <[EMAIL PROTECTED]> wrote:
>>> No that you would want to do this, but simply overwriting a config file would "uncommit" a configuration and make that server think the last committed configuration was whatever is in the file?