Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> [DISCUSS] More new feature backports to 0.94.

Jonathan Hsieh 2013-03-01, 16:31
lars hofhansl 2013-03-01, 19:00
Elliott Clark 2013-03-01, 22:43
Jean-Marc Spaggiari 2013-03-01, 23:12
Enis Söztutar 2013-03-02, 00:55
Jonathan Hsieh 2013-03-01, 23:11
lars hofhansl 2013-03-02, 02:10
Enis Söztutar 2013-03-02, 02:17
Jean-Marc Spaggiari 2013-03-02, 02:25
Ted Yu 2013-03-02, 02:24
Andrew Purtell 2013-03-03, 13:50
Ted 2013-03-03, 14:12
Andrew Purtell 2013-03-03, 14:38
Jean-Marc Spaggiari 2013-03-04, 13:41
Stack 2013-03-04, 21:27
Enis Söztutar 2013-03-04, 22:29
Andrew Purtell 2013-03-05, 01:57
Dave Wang 2013-03-01, 16:38
Lars Hofhansl 2013-03-02, 02:46
Enis Söztutar 2013-03-02, 02:54
Jean-Marc Spaggiari 2013-03-02, 03:12
lars hofhansl 2013-03-02, 03:24
Ted Yu 2013-03-02, 03:30
lars hofhansl 2013-03-02, 03:44
Nicolas Liochon 2013-03-02, 11:43
Ted 2013-03-02, 11:57
Copy link to this message
Re: [DISCUSS] More new feature backports to 0.94.
In general, I have a preference against backporting  features for the
reasons that Enis, Elliott, and Jean-Marc consider valid.  To be clear,
this preference doesn't mean I am -1 to all backports onto the stable
apache branch.  Let's do it case-by-case; my main ask is to make major
backports rare and to make it the norm to require significantly more
evidence of testing than usual.  I will -1 a major backport that lacks this
evidence.  This will come up again in the future.

With the cases Lars proposed -- I prefer #3 (just say no) but find #1 (be
very careful) acceptable given higher level of evidence.  #2 (new release
branch) is onerous -- I'd rather we just get preview-release branches out
more frequently to not have deal with this.  Arguably, the reason we have
the preview-release branches serve the purpose of getting releases out more
frequently and giving a feature time to harden from a few common points.
 My hope is that these preview release will replace what were the 0.x.0 and
0.x.1 releases from  previous versions

So what kind of evidence would I like to see? We can use snapshots case as
an example.

When backporting snapshots was brought up, I actually preferred that we not
backport that feature.  There was demand, so we agreed that we'd do it but
no backport it until it is "rock solid".  Here's evidence to support the
case that the feature and backport is solid:
* It's code history is publicly documented and has been available since
* It's design documentation has been available for even longer.
* The feature is mostly additive and doesn't affect vital paths.
* It was tested against trunk and the later tested against a 0.94 variant
that is closer to the target apache branch.
* The version in the trunk branch has been reviewed by 5 committers.
* Limitations are either documented (please let me know if we should
improve it more) or non-critical.
* Testing and hardening anecdotes have been documented in the original and
backport jira.  There has been some relatively long term testing and fault
injection testing (roughly 4-6 weeks).
* It will be backported in a "big bang" -- all pieces get added or none

This is a level I consider to be stronger than the normal testing expected
for a patch.  Ideally, something at least this level is what I would expect
for other major backports.  Do we agree on that?

For the table locks case, there maybe some of this may be a misperception
in timing from my point of view.  I see a notification about this in jira
which makes me think it is more imminent.   Looking into it, I see that
currently the development and application of the zk table lock feature
isn't complete -- the mechanism is committed but it isn't applied and
integrated into all the operations (split, assign etc still on the way).
 I've asked for documentation and Enis has graciously added a great design
doc that will help reviewers understand it.  I'd love to be able to spend
time system testing to really beating it up or at least have anecdotes from
folks about their efforts on the apache verison.  Finally, I would want to
see this feature come in as a big bang -- get it complete enough in trunk
before backporting the pieces to a stable branch.

I haven't invested time into the online merge backport decision but my
instinct there is to not port the feature as well.  It is less risky since
it is an additive feature but has less reward since we already have a
less-friendly-but-comparable mechanism.  Since merge seems similar to split
(which took a while to get right) testing its correctness in failure cases
at the system level would be a prereq.


On Sat, Mar 2, 2013 at 3:43 AM, Nicolas Liochon <[EMAIL PROTECTED]> wrote:

> New feature is a red herring imho: To me the only question is the
> regression risk.. And a feature can have a much lower regression risk than
> a bug fix. I guess we've all seen a fix for a non critical bug putting down
> a production system. Being able to backport features is a competitive

// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
lars hofhansl 2013-03-02, 16:47
Ted Yu 2013-03-02, 16:14
Jonathan Hsieh 2013-03-02, 16:26
lars hofhansl 2013-03-02, 20:46
Jonathan Hsieh 2013-03-02, 21:49
Stack 2013-03-02, 23:24
lars hofhansl 2013-03-02, 03:23
Lars Hofhansl 2013-03-02, 02:45
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB