I've been reading the archives regarding rolling upgrades. Here's the scenario, given a stable ensemble:

ZK1 <-> ZK2 <-> ZK3

In the above, the zoo.cfg for each server looks like this (pseudo):
server.1=ZK1
server.2=ZK2
server.3=ZK3

I want to add a new server, ZK4. If I understand this correctly, I'd bring up ZK4 with this config:
server.1=ZK1
server.2=ZK2
server.3=ZK3
server.4=ZK4

At this point, though, the configs don't match in the ensemble. How do the ZK instances handle this?

Continuing…

Once ZK4 is up, ZK1 would get the new config and get restarted. Once ZK1 is up, ZK2 gets new config, etc.

At each point of config change, the cluster is in a confused state about the config. Is there code in ZK to handle this?

-JZ

I don't think there is a problem if you do it as you say, or even if you just change the config files of all servers at once and restart them, because a majority of the new config
necessarily intersects with a majority of the old one, so a server who has the latest state will be elected leader.

Alex

> -----Original Message-----
> From: Jordan Zimmerman [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, March 08, 2012 3:31 PM
> To: [EMAIL PROTECTED]
> Subject: Rolling upgrades
>
> I've been reading the archives regarding rolling upgrades. Here's the
> scenario, given a stable ensemble:
>
> ZK1 <-> ZK2 <-> ZK3
>
> In the above, the zoo.cfg for each server looks like this (pseudo):
> server.1=ZK1
> server.2=ZK2
> server.3=ZK3
>
> I want to add a new server, ZK4. If I understand this correctly, I'd
> bring up ZK4 with this config:
> server.1=ZK1
> server.2=ZK2
> server.3=ZK3
> server.4=ZK4
>
> At this point, though, the configs don't match in the ensemble. How do
> the ZK instances handle this?
>
> Continuing...
>
> Once ZK4 is up, ZK1 would get the new config and get restarted. Once
> ZK1 is up, ZK2 gets new config, etc.
>
> At each point of config change, the cluster is in a confused state
> about the config. Is there code in ZK to handle this?
>
> -JZ

AlsoŠ

I thought that ZK ensembles need to be odd in number. How would ZK handle
a temporary state where there is an even number?

-JZ

On 3/8/12 3:39 PM, "Alexander Shraer" <[EMAIL PROTECTED]> wrote:

>I don't think there is a problem if you do it as you say, or even if you
>just change the config files of all servers at once and restart them,
>because a majority of the new config
>necessarily intersects with a majority of the old one, so a server who
>has the latest state will be elected leader.
>
>Alex
>
>> -----Original Message-----
>> From: Jordan Zimmerman [mailto:[EMAIL PROTECTED]]
>> Sent: Thursday, March 08, 2012 3:31 PM
>> To: [EMAIL PROTECTED]
>> Subject: Rolling upgrades
>>
>> I've been reading the archives regarding rolling upgrades. Here's the
>> scenario, given a stable ensemble:
>>
>> ZK1 <-> ZK2 <-> ZK3
>>
>> In the above, the zoo.cfg for each server looks like this (pseudo):
>> server.1=ZK1
>> server.2=ZK2
>> server.3=ZK3
>>
>> I want to add a new server, ZK4. If I understand this correctly, I'd
>> bring up ZK4 with this config:
>> server.1=ZK1
>> server.2=ZK2
>> server.3=ZK3
>> server.4=ZK4
>>
>> At this point, though, the configs don't match in the ensemble. How do
>> the ZK instances handle this?
>>
>> Continuing...
>>
>> Once ZK4 is up, ZK1 would get the new config and get restarted. Once
>> ZK1 is up, ZK2 gets new config, etc.
>>
>> At each point of config change, the cluster is in a confused state
>> about the config. Is there code in ZK to handle this?
>>
>> -JZ
>
>

It won't be any different than a temporary state when one of 3 or 5 nodes
is down.

On Thu, Mar 8, 2012 at 4:10 PM, Jordan Zimmerman <[EMAIL PROTECTED]>wrote:

> AlsoŠ
>
> I thought that ZK ensembles need to be odd in number. How would ZK handle
> a temporary state where there is an even number?
>
> -JZ
>
> On 3/8/12 3:39 PM, "Alexander Shraer" <[EMAIL PROTECTED]> wrote:
>
> >I don't think there is a problem if you do it as you say, or even if you
> >just change the config files of all servers at once and restart them,
> >because a majority of the new config
> >necessarily intersects with a majority of the old one, so a server who
> >has the latest state will be elected leader.
> >
> >Alex
> >
> >> -----Original Message-----
> >> From: Jordan Zimmerman [mailto:[EMAIL PROTECTED]]
> >> Sent: Thursday, March 08, 2012 3:31 PM
> >> To: [EMAIL PROTECTED]
> >> Subject: Rolling upgrades
> >>
> >> I've been reading the archives regarding rolling upgrades. Here's the
> >> scenario, given a stable ensemble:
> >>
> >> ZK1 <-> ZK2 <-> ZK3
> >>
> >> In the above, the zoo.cfg for each server looks like this (pseudo):
> >> server.1=ZK1
> >> server.2=ZK2
> >> server.3=ZK3
> >>
> >> I want to add a new server, ZK4. If I understand this correctly, I'd
> >> bring up ZK4 with this config:
> >> server.1=ZK1
> >> server.2=ZK2
> >> server.3=ZK3
> >> server.4=ZK4
> >>
> >> At this point, though, the configs don't match in the ensemble. How do
> >> the ZK instances handle this?
> >>
> >> Continuing...
> >>
> >> Once ZK4 is up, ZK1 would get the new config and get restarted. Once
> >> ZK1 is up, ZK2 gets new config, etc.
> >>
> >> At each point of config change, the cluster is in a confused state
> >> about the config. Is there code in ZK to handle this?
> >>
> >> -JZ
> >
> >
>
>

Any even number that is greater than half of the configured number of nodes
is fine.

The only *really* bad even number of servers is 0.

On Thu, Mar 8, 2012 at 4:24 PM, Ted Dunning <[EMAIL PROTECTED]> wrote:

> It won't be any different than a temporary state when one of 3 or 5 nodes
> is down.
>
>
> On Thu, Mar 8, 2012 at 4:10 PM, Jordan Zimmerman <[EMAIL PROTECTED]>wrote:
>
>> AlsoŠ
>>
>> I thought that ZK ensembles need to be odd in number. How would ZK handle
>> a temporary state where there is an even number?
>>
>> -JZ
>>
>> On 3/8/12 3:39 PM, "Alexander Shraer" <[EMAIL PROTECTED]> wrote:
>>
>> >I don't think there is a problem if you do it as you say, or even if you
>> >just change the config files of all servers at once and restart them,
>> >because a majority of the new config
>> >necessarily intersects with a majority of the old one, so a server who
>> >has the latest state will be elected leader.
>> >
>> >Alex
>> >
>> >> -----Original Message-----
>> >> From: Jordan Zimmerman [mailto:[EMAIL PROTECTED]]
>> >> Sent: Thursday, March 08, 2012 3:31 PM
>> >> To: [EMAIL PROTECTED]
>> >> Subject: Rolling upgrades
>> >>
>> >> I've been reading the archives regarding rolling upgrades. Here's the
>> >> scenario, given a stable ensemble:
>> >>
>> >> ZK1 <-> ZK2 <-> ZK3
>> >>
>> >> In the above, the zoo.cfg for each server looks like this (pseudo):
>> >> server.1=ZK1
>> >> server.2=ZK2
>> >> server.3=ZK3
>> >>
>> >> I want to add a new server, ZK4. If I understand this correctly, I'd
>> >> bring up ZK4 with this config:
>> >> server.1=ZK1
>> >> server.2=ZK2
>> >> server.3=ZK3
>> >> server.4=ZK4
>> >>
>> >> At this point, though, the configs don't match in the ensemble. How do
>> >> the ZK instances handle this?
>> >>
>> >> Continuing...
>> >>
>> >> Once ZK4 is up, ZK1 would get the new config and get restarted. Once
>> >> ZK1 is up, ZK2 gets new config, etc.
>> >>
>> >> At each point of config change, the cluster is in a confused state
>> >> about the config. Is there code in ZK to handle this?
>> >>
>> >> -JZ
>> >
>> >
>>
>>
>

I don't think they must be an odd number, but it does make sense to have an odd number
because with majority quorums the fault tolerance you get with 3 servers is the same
as the fault tolerance you get with 4 - in both cases you can only tolerate 1 failure.

Alex

> -----Original Message-----
> From: Jordan Zimmerman [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, March 08, 2012 4:10 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Rolling upgrades
>
> AlsoŠ
>
> I thought that ZK ensembles need to be odd in number. How would ZK
> handle
> a temporary state where there is an even number?
>
> -JZ
>
> On 3/8/12 3:39 PM, "Alexander Shraer" <[EMAIL PROTECTED]> wrote:
>
> >I don't think there is a problem if you do it as you say, or even if
> you
> >just change the config files of all servers at once and restart them,
> >because a majority of the new config
> >necessarily intersects with a majority of the old one, so a server who
> >has the latest state will be elected leader.
> >
> >Alex
> >
> >> -----Original Message-----
> >> From: Jordan Zimmerman [mailto:[EMAIL PROTECTED]]
> >> Sent: Thursday, March 08, 2012 3:31 PM
> >> To: [EMAIL PROTECTED]
> >> Subject: Rolling upgrades
> >>
> >> I've been reading the archives regarding rolling upgrades. Here's
> the
> >> scenario, given a stable ensemble:
> >>
> >> ZK1 <-> ZK2 <-> ZK3
> >>
> >> In the above, the zoo.cfg for each server looks like this (pseudo):
> >> server.1=ZK1
> >> server.2=ZK2
> >> server.3=ZK3
> >>
> >> I want to add a new server, ZK4. If I understand this correctly, I'd
> >> bring up ZK4 with this config:
> >> server.1=ZK1
> >> server.2=ZK2
> >> server.3=ZK3
> >> server.4=ZK4
> >>
> >> At this point, though, the configs don't match in the ensemble. How
> do
> >> the ZK instances handle this?
> >>
> >> Continuing...
> >>
> >> Once ZK4 is up, ZK1 would get the new config and get restarted. Once
> >> ZK1 is up, ZK2 gets new config, etc.
> >>
> >> At each point of config change, the cluster is in a confused state
> >> about the config. Is there code in ZK to handle this?
> >>
> >> -JZ
> >
> >