Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk

Copy link to this message
Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk
Sorry, just to be clear "our" and "we" below refer to my employer, nothing
to do with HBase. Please pardon any confusion.

On Tuesday, October 9, 2012, Andrew Purtell wrote:

> Our position on the QJM is we've already "taken delivery" from the feature
> branch and will maintain a private HDFS fork of branch-2 if necessary, i.e.
> we don't have a significant stake in this discussion except at a meta level
> as potential contributors.
> This is a case study of why feature branch development is problematic? A
> would-be contributor can make a significant effort over months and end up
> with a baked result, ready to move on to a perhaps large backlog of other
> work. Others can reasonably be expected meanwhile to triage their attention
> until the merge point. Then, reconsidering design points will be more
> challenging than a design discussion at an earlier time, because there is
> already a significant sunk cost in the current design. What does a feature
> branch buy here over development in situ in trunk (like the BookKeeper JM)?
> Should would-be future contributors consider a feature branch as a viable
> route toward contribution or not?
> On Tuesday, October 9, 2012, Suresh Srinivas wrote:
>> On Mon, Oct 8, 2012 at 6:20 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>> > On Mon, Oct 8, 2012 at 6:01 PM, Suresh Srinivas <[EMAIL PROTECTED]
>> > >wrote:
>> >
>> > > Todd,
>> > >
>> > > As I indicated in my comments on the jira, I think some of the design
>> > > discussions and further simplification of design should happen before
>> the
>> > > merge. See -
>> > >
>> > >
>> >
>> https://issues.apache.org/jira/browse/HDFS-3077?focusedCommentId=13470680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13470680
>> > >
>> > >
>> > I still haven't heard any actual concrete suggestion for a design
>> > improvement. Just abstract notions that "it should be simpler". I could
>> say
>> > the same about several other features that have been done in the past
>> (e.g
>> > federation or YARN) but chose not to because I didn't participate in the
>> > development. I generally have faith that, if several other people spent
>> > several months working on a project, then there must be good reasons for
>> > the complexity that I'm probably overlooking.
>> >
>> I think I have participated enough in this work. Merge time seemed like a
>> good time
>> to review this more thoroughly given this jira has required quite a
>> bit of educating myself and spending a lot of my time on this because of
>> need to
>> understand the complexities/subtletie how paxos and ZAB applies to the
>> namenode
>> journals.
>> Please do not misunderstand that I am trying to block this work from being
>> merged. I am
>> supportive of this as you have seen in my previous response in this
>> thread.
>> All I want
>> to see is if we can conclude the discussions and incorporate some the
>> comments that come
>> up after it.
>> >
>> > Several of us (not just I) spent lots of time on this over the last
>> several
>> > months, with all of the work going on in the open. My issue with this
>> > discussion happening now is that no one saw fit to raise these questions
>> > months ago when the design was first proposed. For example, this most
>> > recent question about whether NewEpoch and PrepareRecovery should be
>> > separate RPCs could have been raised in late June when the code in
>> question
>> > was first introduced as a patch.
>> >
>> > Nor was anything raised when I gave a "heads up" that the branch was
>> > complete and nearly ready for merge ~3 weeks ago. Only once I actually
>> > called a merge vote has this discussion started. That doesn't seem like
>> a
>> > healthy way to do development.
>> >
>> Again Todd, you are reading more than what is intended. As I said earlier
>> merge
>> time is a good time to have complete picture and an opportunity to look at
>> final design
>> and implementation.
>> > What are the criteria, then, for merging? I don't think it's possible to

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)