|
Flavio Junqueira
2012-01-20, 17:25
Alexander Shraer
2012-01-20, 18:47
Benjamin Reed
2012-01-20, 23:29
Thomas Koch
2012-01-21, 06:51
Flavio Junqueira
2012-01-21, 09:02
Ted Dunning
2012-01-21, 09:09
Patrick Hunt
2012-01-26, 00:25
|
-
Branch for reconfiguration (ZOOKEEPER-107)?Flavio Junqueira 2012-01-20, 17:25
This message is to throw the idea and get a sense of what people
think, especially the ones working closely on it like Alex, about creating a branch for the reconfiguration work. The rationale for proposing it is the following. In our experience with Zab last year, we implemented modifications that introduced a bunch of critical bugs. I'm not sure whether we could have been more careful, maybe so, but my perception is that it is not unusual to introduce such bugs once one touches upon a large chunk of core code. Perhaps it would have been better to develop it on the side and test more before merging. For reconfiguration, it sounds like we will be touching a big chunk of core code again, so I wonder if it makes sense to work on a separate branch, test, and merge once we are convinced. I must also say that I understand that merging branches can be quite a pain, so I would completely understand if the general feeling is that it outweighs the benefits. Thanks, -Flavio
-
RE: Branch for reconfiguration (ZOOKEEPER-107)?Alexander Shraer 2012-01-20, 18:47
Hi Flavio,
I've been basically working on my own branch for many months now. The problem is that the main trunk is changing all the time, and of course the merge is difficult as you mentioned. Although my implementation is pretty stable, testing on my own branch doesn't mean that it will work in the trunk. At this point, Ben and I decided to do the merge by partitioning the work into multiple patches, each of which will be well-defined, tested separately and easier to review. So far I have 1 such patch ready - this is Zookeeper-1355 which is waiting to be reviewed. Currently I'm working on 3 more patches, which I'd like to get in before the main algorithm: 1. Separating membership information from the main configuration file. This will not only be needed for reconfiguration but also makes it easier to configure the current system - this way you can have a single membership file and all servers can have a copy. 2. Changing the way the list of peers is stored in QuorumPeer. This eliminates quorumPeers and instead stores the list in a QuorumVerifier. The QuorumVerifier basically becomes a representation of a configuration - it knows who are the voters, who are the observers, etc. This will also solve the current zookeeper bug that allows any connected follower to vote (last week a related bug was solved where observers were counted in the code of the leader, however the same problem appears in many other places in the code, such as leader election and other places in leader code). 3. Each proposal has a set of configurations (QuorumVerifiers) from which it needs an ack in order to commit. Currently each proposal has just one associated configuration. Testing and merging these patches first will make it much much easier to add the reconfiguration algorithm and to test it. Alex > -----Original Message----- > From: Flavio Junqueira [mailto:[EMAIL PROTECTED]] > Sent: Friday, January 20, 2012 9:25 AM > To: [EMAIL PROTECTED] > Subject: Branch for reconfiguration (ZOOKEEPER-107)? > > This message is to throw the idea and get a sense of what people > think, especially the ones working closely on it like Alex, about > creating a branch for the reconfiguration work. The rationale for > proposing it is the following. In our experience with Zab last year, > we implemented modifications that introduced a bunch of critical bugs. > I'm not sure whether we could have been more careful, maybe so, but my > perception is that it is not unusual to introduce such bugs once one > touches upon a large chunk of core code. Perhaps it would have been > better to develop it on the side and test more before merging. > > For reconfiguration, it sounds like we will be touching a big chunk of > core code again, so I wonder if it makes sense to work on a separate > branch, test, and merge once we are convinced. I must also say that I > understand that merging branches can be quite a pain, so I would > completely understand if the general feeling is that it outweighs the > benefits. > > Thanks, > -Flavio
-
Re: Branch for reconfiguration (ZOOKEEPER-107)?Benjamin Reed 2012-01-20, 23:29
certainly the first couple of changes outlined by alex are small and
general (they fix things outside of reconfig) so we can commit those relatively quickly into trunk. perhaps later if the reconfig patch is big and we wanted to get experience with it before we commit to trunk, we can create a branch. ben On Fri, Jan 20, 2012 at 10:47 AM, Alexander Shraer <[EMAIL PROTECTED]> wrote: > Hi Flavio, > > I've been basically working on my own branch for many months now. The problem is that > the main trunk is changing all the time, and of course the merge is difficult as you mentioned. > Although my implementation is pretty stable, testing on my own branch doesn't mean that it will work in the trunk. > > At this point, Ben and I decided to do the merge by partitioning the work into multiple patches, each of which will be well-defined, tested separately and easier to review. So far I have 1 such patch ready - this is Zookeeper-1355 which is waiting to be reviewed. Currently I'm working on 3 more patches, which I'd like to get in before the main algorithm: > > 1. Separating membership information from the main configuration file. This will not only be needed for reconfiguration but also makes it easier to configure the current system - this way you can have a single membership file and all servers can have a copy. > > 2. Changing the way the list of peers is stored in QuorumPeer. This eliminates quorumPeers and instead stores the list in a QuorumVerifier. The QuorumVerifier basically becomes a representation of a configuration - it knows who are the voters, who are the observers, etc. This will also solve the current zookeeper bug that allows any connected follower to vote (last week a related bug was solved where observers were counted in the code of the leader, however the same problem appears in many other places in the code, such as leader election and other places in leader code). > > 3. Each proposal has a set of configurations (QuorumVerifiers) from which it needs an ack in order to commit. Currently each proposal has just one associated configuration. > > Testing and merging these patches first will make it much much easier to add the reconfiguration algorithm and to test it. > > Alex > >> -----Original Message----- >> From: Flavio Junqueira [mailto:[EMAIL PROTECTED]] >> Sent: Friday, January 20, 2012 9:25 AM >> To: [EMAIL PROTECTED] >> Subject: Branch for reconfiguration (ZOOKEEPER-107)? >> >> This message is to throw the idea and get a sense of what people >> think, especially the ones working closely on it like Alex, about >> creating a branch for the reconfiguration work. The rationale for >> proposing it is the following. In our experience with Zab last year, >> we implemented modifications that introduced a bunch of critical bugs. >> I'm not sure whether we could have been more careful, maybe so, but my >> perception is that it is not unusual to introduce such bugs once one >> touches upon a large chunk of core code. Perhaps it would have been >> better to develop it on the side and test more before merging. >> >> For reconfiguration, it sounds like we will be touching a big chunk of >> core code again, so I wonder if it makes sense to work on a separate >> branch, test, and merge once we are convinced. I must also say that I >> understand that merging branches can be quite a pain, so I would >> completely understand if the general feeling is that it outweighs the >> benefits. >> >> Thanks, >> -Flavio >
-
Re: Branch for reconfiguration (ZOOKEEPER-107)?Thomas Koch 2012-01-21, 06:51
Flavio Junqueira:
> This message is to throw the idea and get a sense of what people > think, especially the ones working closely on it like Alex, about > creating a branch for the reconfiguration work. The rationale for > proposing it is the following. In our experience with Zab last year, > we implemented modifications that introduced a bunch of critical bugs. > I'm not sure whether we could have been more careful, maybe so, but my > perception is that it is not unusual to introduce such bugs once one > touches upon a large chunk of core code. Perhaps it would have been > better to develop it on the side and test more before merging. > > For reconfiguration, it sounds like we will be touching a big chunk of > core code again, so I wonder if it makes sense to work on a separate > branch, test, and merge once we are convinced. I must also say that I > understand that merging branches can be quite a pain, so I would > completely understand if the general feeling is that it outweighs the > benefits. > > Thanks, > -Flavio Hi Flavio, I've been working with the Git mirror of ZooKeeper and a Git branch of my proposed patches for two months, constantly (daily) rebasing my work against trunk. Every commit in my branch reflected a jira issue. If an issue got accepted into trunk, I rebased by Git branch on top of the new trunk but without the respectiv commit: https://github.com/thkoch2001/zookeeper/commits/proposed_patches It's not as hard as it sounds and it keeps your changes current instead of a merge nightmare after weeks of separated development in trunk and in your branch. I believe, that there is still a lot of valuable refactoring in the above branch that would make any ZK development a lot easier. A Subversion branch won't help you, I think. Best regards, Thomas Koch, http://www.koch.ro
-
Re: Branch for reconfiguration (ZOOKEEPER-107)?Flavio Junqueira 2012-01-21, 09:02
I like the idea of a github repository to share code changes in
progress. Pat has also pointed it out offline. -Flavio On Jan 21, 2012, at 7:51 AM, Thomas Koch wrote: > Flavio Junqueira: >> This message is to throw the idea and get a sense of what people >> think, especially the ones working closely on it like Alex, about >> creating a branch for the reconfiguration work. The rationale for >> proposing it is the following. In our experience with Zab last year, >> we implemented modifications that introduced a bunch of critical >> bugs. >> I'm not sure whether we could have been more careful, maybe so, but >> my >> perception is that it is not unusual to introduce such bugs once one >> touches upon a large chunk of core code. Perhaps it would have been >> better to develop it on the side and test more before merging. >> >> For reconfiguration, it sounds like we will be touching a big chunk >> of >> core code again, so I wonder if it makes sense to work on a separate >> branch, test, and merge once we are convinced. I must also say that I >> understand that merging branches can be quite a pain, so I would >> completely understand if the general feeling is that it outweighs the >> benefits. >> >> Thanks, >> -Flavio > > Hi Flavio, > > I've been working with the Git mirror of ZooKeeper and a Git branch > of my > proposed patches for two months, constantly (daily) rebasing my work > against > trunk. Every commit in my branch reflected a jira issue. If an issue > got > accepted into trunk, I rebased by Git branch on top of the new trunk > but > without the respectiv commit: > > https://github.com/thkoch2001/zookeeper/commits/proposed_patches > > It's not as hard as it sounds and it keeps your changes current > instead of a > merge nightmare after weeks of separated development in trunk and in > your > branch. > > I believe, that there is still a lot of valuable refactoring in the > above > branch that would make any ZK development a lot easier. > > A Subversion branch won't help you, I think. > > Best regards, > > Thomas Koch, http://www.koch.ro flavio junqueira research scientist [EMAIL PROTECTED] direct +34 93-183-8828 avinguda diagonal 177, 8th floor, barcelona, 08018, es phone (408) 349 3300 fax (408) 349 3301
-
Re: Branch for reconfiguration (ZOOKEEPER-107)?Ted Dunning 2012-01-21, 09:09
Works great.
See https://github.com/tdunning/zookeeper for work on ZK-1366 including tar distros for testing. On Sat, Jan 21, 2012 at 1:02 AM, Flavio Junqueira <[EMAIL PROTECTED]> wrote: > I like the idea of a github repository to share code changes in progress. > Pat has also pointed it out offline. > > -Flavio > > On Jan 21, 2012, at 7:51 AM, Thomas Koch wrote: > > Flavio Junqueira: >> >>> This message is to throw the idea and get a sense of what people >>> think, especially the ones working closely on it like Alex, about >>> creating a branch for the reconfiguration work. The rationale for >>> proposing it is the following. In our experience with Zab last year, >>> we implemented modifications that introduced a bunch of critical bugs. >>> I'm not sure whether we could have been more careful, maybe so, but my >>> perception is that it is not unusual to introduce such bugs once one >>> touches upon a large chunk of core code. Perhaps it would have been >>> better to develop it on the side and test more before merging. >>> >>> For reconfiguration, it sounds like we will be touching a big chunk of >>> core code again, so I wonder if it makes sense to work on a separate >>> branch, test, and merge once we are convinced. I must also say that I >>> understand that merging branches can be quite a pain, so I would >>> completely understand if the general feeling is that it outweighs the >>> benefits. >>> >>> Thanks, >>> -Flavio >>> >> >> Hi Flavio, >> >> I've been working with the Git mirror of ZooKeeper and a Git branch of my >> proposed patches for two months, constantly (daily) rebasing my work >> against >> trunk. Every commit in my branch reflected a jira issue. If an issue got >> accepted into trunk, I rebased by Git branch on top of the new trunk but >> without the respectiv commit: >> >> https://github.com/thkoch2001/**zookeeper/commits/proposed_**patches<https://github.com/thkoch2001/zookeeper/commits/proposed_patches> >> >> It's not as hard as it sounds and it keeps your changes current instead >> of a >> merge nightmare after weeks of separated development in trunk and in your >> branch. >> >> I believe, that there is still a lot of valuable refactoring in the above >> branch that would make any ZK development a lot easier. >> >> A Subversion branch won't help you, I think. >> >> Best regards, >> >> Thomas Koch, http://www.koch.ro >> > > flavio > junqueira > > research scientist > > [EMAIL PROTECTED] > direct +34 93-183-8828 > > avinguda diagonal 177, 8th floor, barcelona, 08018, es > phone (408) 349 3300 fax (408) 349 3301 > >
-
Re: Branch for reconfiguration (ZOOKEEPER-107)?Patrick Hunt 2012-01-26, 00:25
On Sat, Jan 21, 2012 at 1:02 AM, Flavio Junqueira <[EMAIL PROTECTED]> wrote:
> I like the idea of a github repository to share code changes in progress. > Pat has also pointed it out offline. I personally use git (and github) and it works well for me. However my concern in this case is that if we don't integrate directly into the trunk there's a chance it will get orphaned. Not fair for Alex, plus we miss out on the functionality. I'd rather see a number of smaller patches that are easy to review, verify and test committed to trunk. In no case do I want to see a big patch or a single merge from some branch (svn, git or otw). All changes must ensure backward compatibility. See what Todd did here for a great example of how a large refactoring was handled (relatively) recently in hdfs: https://issues.apache.org/jira/browse/HDFS-1073 Given this is an optional feature it's not high risk (if it's broken don't use it). Although there's quite a bit of debt built up in the code, a major concern of mine is that we'll break existing functionality in some subtle way and miss it. Patrick |