Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk


Copy link to this message
-
Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk
On Mon, Oct 8, 2012 at 6:01 PM, Suresh Srinivas <[EMAIL PROTECTED]>wrote:

> Todd,
>
> As I indicated in my comments on the jira, I think some of the design
> discussions and further simplification of design should happen before the
> merge. See -
>
> https://issues.apache.org/jira/browse/HDFS-3077?focusedCommentId=13470680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13470680
>
>
I still haven't heard any actual concrete suggestion for a design
improvement. Just abstract notions that "it should be simpler". I could say
the same about several other features that have been done in the past (e.g
federation or YARN) but chose not to because I didn't participate in the
development. I generally have faith that, if several other people spent
several months working on a project, then there must be good reasons for
the complexity that I'm probably overlooking.

Several of us (not just I) spent lots of time on this over the last several
months, with all of the work going on in the open. My issue with this
discussion happening now is that no one saw fit to raise these questions
months ago when the design was first proposed. For example, this most
recent question about whether NewEpoch and PrepareRecovery should be
separate RPCs could have been raised in late June when the code in question
was first introduced as a patch.

Nor was anything raised when I gave a "heads up" that the branch was
complete and nearly ready for merge ~3 weeks ago. Only once I actually
called a merge vote has this discussion started. That doesn't seem like a
healthy way to do development.
> I understand that you have spent a lot of time working on this. You have
> indicated that you do not want to make any further design improvements. I
> am willing to help by doing any improvements that comes out of the
> discussions on the jira in 3077 branch and keep it up to date. I am also
> willing to merge this branch into trunk. So my vote is to hold off the
> merge until the discussions complete.
>

What are the criteria, then, for merging? I don't think it's possible to
definitively "prove" that a design is "simple". At some level it's a matter
of taste. So if the current design works, and is tested, and has people
signed up to support it and run it, why not merge? Just like any other part
of HDFS, we can continue to _improve_ it after it is in trunk. There are
many features that we commit and then improve later on when someone comes
up with a way to simplify it.

To be clear about the current status of the vote: you are officially
vetoing the merge?

Thanks
-Todd

 On Mon, Oct 8, 2012 at 5:46 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>
> > Hi Sanjay,
> >
> > The 7 extra days you requested beyond the original 7-day merge vote have
> > now elapsed, and we have the requisite three binding +1s to merge.
> >
> > I'll plan to merge this late tonight unless there are any vetoes in the
> > meantime.
> >
> > Of course we can continue to discuss the design and improve the clarity
> of
> > the documentation after it's in trunk, and if there's some kind of bug
> I'll
> > treat it as highest priority even after the merge.
> >
> > Thanks
> > -Todd
> >
> >
> > On Mon, Oct 1, 2012 at 10:55 AM, sanjay Radia <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Todd,
> > >   Even though this work was under development over a period of time,
> > >  during its development it was not clear when the design was fairly
> > stable
> > > to begin a thorough review. Hence the time of merge is when the real
> > review
> > > happens in such large projects.
> > >
> > > I have already indicated on the jira that i do not have any
> philosophical
> > > objection to this work being in HDFS - hence this should not be a worry
> > on
> > > your part.
> > >
> > > The extra week will result in a more through review (hopefully this
> will
> > > have a side effect of perhaps easing Konstanine's concern about
> > > HDFS adding such complex code).
> > >
> > > Lets plan to do the merge next monday.

Todd Lipcon
Software Engineer, Cloudera
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB