Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk


Copy link to this message
-
Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk
Our position on the QJM is we've already "taken delivery" from the feature
branch and will maintain a private HDFS fork of branch-2 if necessary, i.e.
we don't have a significant stake in this discussion except at a meta level
as potential contributors.

This is a case study of why feature branch development is problematic? A
would-be contributor can make a significant effort over months and end up
with a baked result, ready to move on to a perhaps large backlog of other
work. Others can reasonably be expected meanwhile to triage their attention
until the merge point. Then, reconsidering design points will be more
challenging than a design discussion at an earlier time, because there is
already a significant sunk cost in the current design. What does a feature
branch buy here over development in situ in trunk (like the BookKeeper JM)?
Should would-be future contributors consider a feature branch as a viable
route toward contribution or not?

On Tuesday, October 9, 2012, Suresh Srinivas wrote:

> On Mon, Oct 8, 2012 at 6:20 PM, Todd Lipcon <[EMAIL PROTECTED]<javascript:;>>
> wrote:
>
> > On Mon, Oct 8, 2012 at 6:01 PM, Suresh Srinivas <[EMAIL PROTECTED]<javascript:;>
> > >wrote:
> >
> > > Todd,
> > >
> > > As I indicated in my comments on the jira, I think some of the design
> > > discussions and further simplification of design should happen before
> the
> > > merge. See -
> > >
> > >
> >
> https://issues.apache.org/jira/browse/HDFS-3077?focusedCommentId=13470680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13470680
> > >
> > >
> > I still haven't heard any actual concrete suggestion for a design
> > improvement. Just abstract notions that "it should be simpler". I could
> say
> > the same about several other features that have been done in the past
> (e.g
> > federation or YARN) but chose not to because I didn't participate in the
> > development. I generally have faith that, if several other people spent
> > several months working on a project, then there must be good reasons for
> > the complexity that I'm probably overlooking.
> >
>
> I think I have participated enough in this work. Merge time seemed like a
> good time
> to review this more thoroughly given this jira has required quite a
> bit of educating myself and spending a lot of my time on this because of
> need to
> understand the complexities/subtletie how paxos and ZAB applies to the
> namenode
> journals.
>
> Please do not misunderstand that I am trying to block this work from being
> merged. I am
> supportive of this as you have seen in my previous response in this thread.
> All I want
> to see is if we can conclude the discussions and incorporate some the
> comments that come
> up after it.
>
> >
> > Several of us (not just I) spent lots of time on this over the last
> several
> > months, with all of the work going on in the open. My issue with this
> > discussion happening now is that no one saw fit to raise these questions
> > months ago when the design was first proposed. For example, this most
> > recent question about whether NewEpoch and PrepareRecovery should be
> > separate RPCs could have been raised in late June when the code in
> question
> > was first introduced as a patch.
> >
> > Nor was anything raised when I gave a "heads up" that the branch was
> > complete and nearly ready for merge ~3 weeks ago. Only once I actually
> > called a merge vote has this discussion started. That doesn't seem like a
> > healthy way to do development.
> >
>
> Again Todd, you are reading more than what is intended. As I said earlier
> merge
> time is a good time to have complete picture and an opportunity to look at
> final design
> and implementation.
>
>
> > What are the criteria, then, for merging? I don't think it's possible to
> > definitively "prove" that a design is "simple". At some level it's a
> matter
> > of taste. So if the current design works, and is tested, and has people
> > signed up to support it and run it, why not merge? Just like any other
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB