Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # dev - Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk


+
sanjay Radia 2012-09-28, 19:06
+
Todd Lipcon 2012-09-28, 22:02
+
sanjay Radia 2012-10-01, 17:55
+
Todd Lipcon 2012-10-09, 00:46
+
Suresh Srinivas 2012-10-09, 01:01
+
Todd Lipcon 2012-10-09, 01:20
+
Suresh Srinivas 2012-10-09, 02:21
+
Eli Collins 2012-10-09, 04:11
+
sanjay Radia 2012-10-09, 20:44
+
sanjay Radia 2012-10-11, 02:17
+
Suresh Srinivas 2012-10-11, 04:10
+
Todd Lipcon 2012-10-11, 08:12
Copy link to this message
-
Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk
Andrew Purtell 2012-10-09, 03:03
Our position on the QJM is we've already "taken delivery" from the feature
branch and will maintain a private HDFS fork of branch-2 if necessary, i.e.
we don't have a significant stake in this discussion except at a meta level
as potential contributors.

This is a case study of why feature branch development is problematic? A
would-be contributor can make a significant effort over months and end up
with a baked result, ready to move on to a perhaps large backlog of other
work. Others can reasonably be expected meanwhile to triage their attention
until the merge point. Then, reconsidering design points will be more
challenging than a design discussion at an earlier time, because there is
already a significant sunk cost in the current design. What does a feature
branch buy here over development in situ in trunk (like the BookKeeper JM)?
Should would-be future contributors consider a feature branch as a viable
route toward contribution or not?

On Tuesday, October 9, 2012, Suresh Srinivas wrote:

> On Mon, Oct 8, 2012 at 6:20 PM, Todd Lipcon <[EMAIL PROTECTED]<javascript:;>>
> wrote:
>
> > On Mon, Oct 8, 2012 at 6:01 PM, Suresh Srinivas <[EMAIL PROTECTED]<javascript:;>
> > >wrote:
> >
> > > Todd,
> > >
> > > As I indicated in my comments on the jira, I think some of the design
> > > discussions and further simplification of design should happen before
> the
> > > merge. See -
> > >
> > >
> >
> https://issues.apache.org/jira/browse/HDFS-3077?focusedCommentId=13470680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13470680
> > >
> > >
> > I still haven't heard any actual concrete suggestion for a design
> > improvement. Just abstract notions that "it should be simpler". I could
> say
> > the same about several other features that have been done in the past
> (e.g
> > federation or YARN) but chose not to because I didn't participate in the
> > development. I generally have faith that, if several other people spent
> > several months working on a project, then there must be good reasons for
> > the complexity that I'm probably overlooking.
> >
>
> I think I have participated enough in this work. Merge time seemed like a
> good time
> to review this more thoroughly given this jira has required quite a
> bit of educating myself and spending a lot of my time on this because of
> need to
> understand the complexities/subtletie how paxos and ZAB applies to the
> namenode
> journals.
>
> Please do not misunderstand that I am trying to block this work from being
> merged. I am
> supportive of this as you have seen in my previous response in this thread.
> All I want
> to see is if we can conclude the discussions and incorporate some the
> comments that come
> up after it.
>
> >
> > Several of us (not just I) spent lots of time on this over the last
> several
> > months, with all of the work going on in the open. My issue with this
> > discussion happening now is that no one saw fit to raise these questions
> > months ago when the design was first proposed. For example, this most
> > recent question about whether NewEpoch and PrepareRecovery should be
> > separate RPCs could have been raised in late June when the code in
> question
> > was first introduced as a patch.
> >
> > Nor was anything raised when I gave a "heads up" that the branch was
> > complete and nearly ready for merge ~3 weeks ago. Only once I actually
> > called a merge vote has this discussion started. That doesn't seem like a
> > healthy way to do development.
> >
>
> Again Todd, you are reading more than what is intended. As I said earlier
> merge
> time is a good time to have complete picture and an opportunity to look at
> final design
> and implementation.
>
>
> > What are the criteria, then, for merging? I don't think it's possible to
> > definitively "prove" that a design is "simple". At some level it's a
> matter
> > of taste. So if the current design works, and is tested, and has people
> > signed up to support it and run it, why not merge? Just like any other
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
+
Suresh Srinivas 2012-10-09, 03:32
+
Andrew Purtell 2012-10-09, 03:09
+
Eli Collins 2012-10-09, 01:48
+
sanjay Radia 2012-09-28, 19:03
+
Stack 2012-09-27, 16:59
+
Todd Lipcon 2012-09-27, 20:29
+
Suresh Srinivas 2012-09-27, 16:47
+
Andrew Purtell 2012-09-27, 09:29