Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - [DISCUSS] More new feature backports to 0.94.


Copy link to this message
-
Re: [DISCUSS] More new feature backports to 0.94.
lars hofhansl 2013-03-02, 16:47
I on the same page as N here.

There are "features" that make HBase more stable. Table locks (when implemented correctly and finished of course) could be such a feature.
I do not understand how 0.95 plays into this. There is no rolling upgrade path from 0.94 to 0.95 and 0.95 is not a stable release.

-- Lars

________________________________
 From: Jonathan Hsieh <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Cc: lars hofhansl <[EMAIL PROTECTED]>
Sent: Saturday, March 2, 2013 7:36 AM
Subject: Re: [DISCUSS] More new feature backports to 0.94.
 

In general, I have a preference against backporting  features for the reasons that Enis, Elliott, and Jean-Marc consider valid.  To be clear, this preference doesn't mean I am -1 to all backports onto the stable apache branch.  Let's do it case-by-case; my main ask is to make major backports rare and to make it the norm to require significantly more evidence of testing than usual.  I will -1 a major backport that lacks this evidence.  This will come up again in the future.

With the cases Lars proposed -- I prefer #3 (just say no) but find #1 (be very careful) acceptable given higher level of evidence.  #2 (new release branch) is onerous -- I'd rather we just get preview-release branches out more frequently to not have deal with this.  Arguably, the reason we have the preview-release branches serve the purpose of getting releases out more frequently and giving a feature time to harden from a few common points.  My hope is that these preview release will replace what were the 0.x.0 and 0.x.1 releases from  previous versions 

So what kind of evidence would I like to see? We can use snapshots case as an example.
When backporting snapshots was brought up, I actually preferred that we not backport that feature.  There was demand, so we agreed that we'd do it but no backport it until it is "rock solid".  Here's evidence to support the case that the feature and backport is solid:

* It's code history is publicly documented and has been available since December.
* It's design documentation has been available for even longer.  
* The feature is mostly additive and doesn't affect vital paths.
* It was tested against trunk and the later tested against a 0.94 variant that is closer to the target apache branch.  
* The version in the trunk branch has been reviewed by 5 committers.
* Limitations are either documented (please let me know if we should improve it more) or non-critical.
* Testing and hardening anecdotes have been documented in the original and backport jira.  There has been some relatively long term testing and fault injection testing (roughly 4-6 weeks).
* It will be backported in a "big bang" -- all pieces get added or none will.
 
This is a level I consider to be stronger than the normal testing expected for a patch.  Ideally, something at least this level is what I would expect for other major backports.  Do we agree on that?

For the table locks case, there maybe some of this may be a misperception in timing from my point of view.  I see a notification about this in jira which makes me think it is more imminent.   Looking into it, I see that currently the development and application of the zk table lock feature isn't complete -- the mechanism is committed but it isn't applied and integrated into all the operations (split, assign etc still on the way).  I've asked for documentation and Enis has graciously added a great design doc that will help reviewers understand it.  I'd love to be able to spend time system testing to really beating it up or at least have anecdotes from folks about their efforts on the apache verison.  Finally, I would want to see this feature come in as a big bang -- get it complete enough in trunk before backporting the pieces to a stable branch.

I haven't invested time into the online merge backport decision but my instinct there is to not port the feature as well.  It is less risky since it is an additive feature but has less reward since we already have a less-friendly-but-comparable mechanism.  Since merge seems similar to split (which took a while to get right) testing its correctness in failure cases at the system level would be a prereq.

Jon.
On Sat, Mar 2, 2013 at 3:43 AM, Nicolas Liochon <[EMAIL PROTECTED]> wrote:

New feature is a red herring imho: To me the only question is the
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera

// [EMAIL PROTECTED]