Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # dev >> Branch for HDFS-1073 and related work


+
Todd Lipcon 2011-03-28, 20:14
+
Dhruba Borthakur 2011-03-28, 20:40
+
Jakob Homan 2011-03-28, 23:39
Copy link to this message
-
Re: Branch for HDFS-1073 and related work
+1 for creating a branch.
Agree with Jakob this should not mean less intensive reviewing.
--Konstantin

On Mon, Mar 28, 2011 at 4:39 PM, Jakob Homan <[EMAIL PROTECTED]> wrote:

> Doing this work on a branch makes sense.  +1.
>
> However, the patches that have been committed so far required
> extensive review and revision, and in one case, an addendum patch.
> Additionally, the patches in related JIRAs such as HDFS-1557 and
> HDFS-1572 have required multiple revisions and corrections.  While
> it's up to each committer which +1 they're willing to accept, I don't
> see any harm in keeping our current standards for review, considering
> the importance and difficulty of this work.  In fact, since moving the
> work to a branch will also move it off of many reviewers' radar, it
> may even be reasonable to increase the scrutiny.  Reviewing giant
> branch merges is difficult and, I believe, more error-prone than
> reviewing smaller packets of work.  So far these patches have received
> timely reviews, if this does not turn out to be the case, let other
> committers know so we can do our part.
> -jg
>
>
> On Mon, Mar 28, 2011 at 1:40 PM, Dhruba Borthakur <[EMAIL PROTECTED]>
> wrote:
> > +1. I think this will be very helpful in moving the design forward
> quickly.
> >
> > -dhruba
> >
> >
> > On Mon, Mar 28, 2011 at 1:14 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
> >
> >> Hi all,
> >>
> >> I discussed this with a couple folks over the weekend who are involved
> in
> >> the project, but wanted to let the dev community at large know:
> >>
> >> I'm planning on creating a new SVN branch for HDFS-1073 and its
> subtasks.
> >> For those not aware, HDFS-1073 is a rethinking of how the NN, 2NN, and
> >> BackupNode store images and edit logs on disk. This will help make HA
> more
> >> manageable down the line and has a lot of operational benefits as
> described
> >> in the JIRA. The "related work" is the addition of transaction IDs to
> the
> >> persistent storage of the NN, and some refactoring in the edit log
> >> subsystem.
> >>
> >> The reasoning behind creating a branch is that, since this is a fairly
> >> large
> >> change, it is easier to develop through a number of subtasks. But at
> some
> >> of
> >> the intermediate points, various components will be temporarily broken.
> >> Developing on a branch will allow us to make incremental progress
> without
> >> worrying about keeping all tests green after every change. We will of
> >> course
> >> make sure all tests pass before merging back into trunk. There will also
> be
> >> another opportunity to review before the merge into trunk. This is the
> same
> >> development methodology as was done for the 0.21 Append work and is now
> >> being used for Federation.
> >>
> >> Given that there will be another opportunity to review these changes
> before
> >> merging into trunk, I would also like to propose that Ivan Kelly be
> granted
> >> the ability to "+1" patches on this branch despite not being an HDFS
> >> committer. Ivan is actively contributing on this project and understands
> >> the
> >> code well.
> >>
> >> Unless there are any objections, I will create this branch and an
> >> associated
> >> Fix Version on JIRA tonight.
> >>
> >> Thanks!
> >> -Todd
> >> --
> >> Todd Lipcon
> >> Software Engineer, Cloudera
> >>
> >
> >
> >
> > --
> > Connect to me at http://www.facebook.com/dhruba
> >
>
+
Todd Lipcon 2011-03-29, 06:18
+
Todd Lipcon 2011-03-30, 01:51
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB