Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # dev - Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk


Copy link to this message
-
Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk
Andrew Purtell 2012-10-09, 03:09
Sorry, just to be clear "our" and "we" below refer to my employer, nothing
to do with HBase. Please pardon any confusion.

On Tuesday, October 9, 2012, Andrew Purtell wrote:

> Our position on the QJM is we've already "taken delivery" from the feature
> branch and will maintain a private HDFS fork of branch-2 if necessary, i.e.
> we don't have a significant stake in this discussion except at a meta level
> as potential contributors.
>
> This is a case study of why feature branch development is problematic? A
> would-be contributor can make a significant effort over months and end up
> with a baked result, ready to move on to a perhaps large backlog of other
> work. Others can reasonably be expected meanwhile to triage their attention
> until the merge point. Then, reconsidering design points will be more
> challenging than a design discussion at an earlier time, because there is
> already a significant sunk cost in the current design. What does a feature
> branch buy here over development in situ in trunk (like the BookKeeper JM)?
> Should would-be future contributors consider a feature branch as a viable
> route toward contribution or not?
>
> On Tuesday, October 9, 2012, Suresh Srinivas wrote:
>
>> On Mon, Oct 8, 2012 at 6:20 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>>
>> > On Mon, Oct 8, 2012 at 6:01 PM, Suresh Srinivas <[EMAIL PROTECTED]
>> > >wrote:
>> >
>> > > Todd,
>> > >
>> > > As I indicated in my comments on the jira, I think some of the design
>> > > discussions and further simplification of design should happen before
>> the
>> > > merge. See -
>> > >
>> > >
>> >
>> https://issues.apache.org/jira/browse/HDFS-3077?focusedCommentId=13470680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13470680
>> > >
>> > >
>> > I still haven't heard any actual concrete suggestion for a design
>> > improvement. Just abstract notions that "it should be simpler". I could
>> say
>> > the same about several other features that have been done in the past
>> (e.g
>> > federation or YARN) but chose not to because I didn't participate in the
>> > development. I generally have faith that, if several other people spent
>> > several months working on a project, then there must be good reasons for
>> > the complexity that I'm probably overlooking.
>> >
>>
>> I think I have participated enough in this work. Merge time seemed like a
>> good time
>> to review this more thoroughly given this jira has required quite a
>> bit of educating myself and spending a lot of my time on this because of
>> need to
>> understand the complexities/subtletie how paxos and ZAB applies to the
>> namenode
>> journals.
>>
>> Please do not misunderstand that I am trying to block this work from being
>> merged. I am
>> supportive of this as you have seen in my previous response in this
>> thread.
>> All I want
>> to see is if we can conclude the discussions and incorporate some the
>> comments that come
>> up after it.
>>
>> >
>> > Several of us (not just I) spent lots of time on this over the last
>> several
>> > months, with all of the work going on in the open. My issue with this
>> > discussion happening now is that no one saw fit to raise these questions
>> > months ago when the design was first proposed. For example, this most
>> > recent question about whether NewEpoch and PrepareRecovery should be
>> > separate RPCs could have been raised in late June when the code in
>> question
>> > was first introduced as a patch.
>> >
>> > Nor was anything raised when I gave a "heads up" that the branch was
>> > complete and nearly ready for merge ~3 weeks ago. Only once I actually
>> > called a merge vote has this discussion started. That doesn't seem like
>> a
>> > healthy way to do development.
>> >
>>
>> Again Todd, you are reading more than what is intended. As I said earlier
>> merge
>> time is a good time to have complete picture and an opportunity to look at
>> final design
>> and implementation.
>>
>>
>> > What are the criteria, then, for merging? I don't think it's possible to

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)