Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> Re: Heads up - 2.0.5-beta

Suresh Srinivas 2013-04-26, 01:36
Arun C Murthy 2013-04-26, 18:02
Konstantin Shvachko 2013-04-26, 23:34
Arun C Murthy 2013-04-27, 01:06
Konstantin Shvachko 2013-04-30, 23:28
Arun C Murthy 2013-05-01, 20:15
Konstantin Shvachko 2013-05-01, 23:08
Arun C Murthy 2013-05-02, 01:24
Chris Douglas 2013-05-02, 07:07
Konstantin Shvachko 2013-05-02, 09:11
Chris Douglas 2013-05-03, 00:53
Konstantin Shvachko 2013-05-02, 09:08
Arun C Murthy 2013-05-02, 22:05
Konstantin Shvachko 2013-05-03, 08:06
Copy link to this message
Re: Heads up - 2.0.5-beta
I agree that "destructive" is not the correct word to describe features
like snapshots and windows support.  However, I also agree with Konstantin
that any large feature will have a destabilizing effect on the code base,
even if it is done on a branch and thoroughly tested before being merged
in. HDFS HA from what I have seen and heard is rock solid, but it took a
while to get there even after it was merged into branch-2. And we all know
how long YARN and MRv2 have taken to stabilize.

I also agree that no one individual is able to police all of Hadoop.  We
have to rely on the committers to make sure that what is placed in a
branch is appropriate for that branch in preparation for a release.  As a
community we need to decided what the goals of a branch are so that I as a
committer can know what is and is not appropriate to be placed in that
branch.  This is the reason why we are discussing API and binary
compatibility. This is the reason why I support having a vote for a
release plan.  The question for the community comes down to do we want to
release quickly and often off of trunk trying hard to maintain
compatibility between releases or do we want to follow what we have done
up to now where a single branch goes into stabilization, trunk gets
anything that is not "compatible" with that branch, and it takes a huge
effort to switch momentum from one branch to another.  Up to this point we
have almost successfully done this switch once, from 1.0 to 2.0. I have a
hard time believing that we are going to do this again for another 5 years.

There is nothing preventing the community from letting each organization
decide what they want to do and we end up with both.  But this results in
fragmentation of the community, and makes it difficult for those trying to
stabilize a release because there is no critical mass of individuals using
and testing that branch.  It also results in the scrambling we are seeing
now to try and revert the incompatibles between 1.0 and 2.0 that were
introduced in the years between these releases.  If we are going to do the
same and make 3.0 compatible with 2.0 when the switch comes, why do we
even allow any incompatible changes in at all?  It just feels like trunk
is a place to put tech debt that we are going to try and revert later.  I
personally like the Linux and BSD models, where there is a new feature
merge window and any new features can come in, then the entire community
works together to stabilize the release before going on the the next merge
window.  If the release does not stabilize quickly the next merge window
gets pushed back. I realize this is very different from the current model
and is not likely to receive a lot of support, but it has worked for them
for a long time, and they have code bases just as large as Hadoop and even
larger and more diverse communities.

I am +1 for Konstantin's release plan and will vote as such on that thread.


On 5/3/13 3:06 AM, "Konstantin Shvachko" <[EMAIL PROTECTED]> wrote:

>Hi Arun and Suresh,
>I am glad my choice of words attracted your attention. I consider this
>important for the project otherwise I wouldn't waste everybody's time.
>You tend reacting on a latest message taken out of context, which does not
>reveal full picture.
>I'll try here to summarize my proposal and motivation expressed earlier in
>these two threads:
>I am advocating
>1. to make 2.0.5 a release that will
>    a) make any necessary changes so that Hadoop APIs could be fixed after
>    b) fix bugs: internal and those important for stabilizing downstream
>2. Release 2.1.0 stable. I.e. both with stable APIs and stable code base.
>3. Produce a series of feature releases. Potentially catching up with the
>state of trunk.
>4. Release from trunk afterwards.
>The main motivation to minimize changes in 2.0.5 is to let Hadoop users
>the downstream projects, that is the Hadoop community, to start adapting
Konstantin Shvachko 2013-05-01, 19:13
Arpit Agarwal 2013-04-26, 23:52
Amir Sanjar 2013-04-26, 02:23
Steve Loughran 2013-04-29, 22:37
Amir Sanjar 2013-04-30, 02:05
Steve Loughran 2013-04-30, 16:48
Amir Sanjar 2013-04-30, 17:09
Roman Shaposhnik 2013-04-26, 02:31
Arun C Murthy 2013-04-26, 18:15
Eli Collins 2013-04-26, 19:07
Suresh Srinivas 2013-04-26, 21:42
Eli Collins 2013-04-26, 22:09
Luke Lu 2013-04-26, 20:37
Arun C Murthy 2013-04-27, 01:27
Konstantin Shvachko 2013-04-26, 23:00
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB