Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # dev - Re: [VOTE] - Release 2.0.5-beta


+
Suresh Srinivas 2013-05-15, 18:02
+
Arpit Gupta 2013-05-15, 19:54
+
Vinod Kumar Vavilapalli 2013-05-15, 18:59
+
Vinod Kumar Vavilapalli 2013-05-15, 19:11
+
Arun C Murthy 2013-05-15, 19:19
+
Karthik Kambatla 2013-05-15, 18:23
+
Alejandro Abdelnur 2013-05-15, 18:32
+
Vinod Kumar Vavilapalli 2013-05-15, 19:19
+
Bikas Saha 2013-05-15, 20:16
+
Matt Foley 2013-05-15, 20:23
+
Amir Sanjar 2013-05-15, 18:14
+
Vinod Kumar Vavilapalli 2013-05-15, 18:06
+
Nathan Roberts 2013-05-16, 18:28
+
Vinod Kumar Vavilapalli 2013-05-17, 07:06
+
eric baldeschwieler 2013-05-15, 20:20
+
Sandy Ryza 2013-05-15, 20:25
+
Matt Foley 2013-05-15, 20:29
+
Eli Collins 2013-05-15, 20:43
+
Zhijie Shen 2013-05-15, 20:47
+
Steve Loughran 2013-05-15, 21:21
+
Chris Douglas 2013-05-15, 22:27
+
Arun C Murthy 2013-05-15, 23:44
+
Matt Foley 2013-05-15, 23:56
+
Konstantin Shvachko 2013-05-15, 22:36
+
Roman Shaposhnik 2013-05-15, 20:29
+
Matt Foley 2013-05-15, 20:36
+
Roman Shaposhnik 2013-05-15, 20:56
+
Vinod Kumar Vavilapalli 2013-05-15, 21:14
+
Roman Shaposhnik 2013-05-15, 21:54
Copy link to this message
-
Re: [VOTE] - Release 2.0.5-beta
Konstantin Boudnik 2013-05-15, 23:17
Indeed. I think the root of the issue is deeper. ASF software practices are
great to deal with isolated, relatively contained projects like httpd,
libreoffice, trac, etc. However, Hadoop based stack - essentially, software
aimed at enterprises with bigger scale operations - is a different animal,
that requires balancing of a huge number of moving parts and an unbroken
flow of feedback up the stream. Anyone who have delivered any enterprise
grade software system knows perfectly well how hard is that.

However, in the environment where a release pushed out in the rush
(essentially causing DOA issues downstream), these are got fixed in consequent
releases.  That ironically is likely to contain some other DOAs because an
integration testing - and I mean real world integration system testing - is
done by this small project, that is treated like a toy for adolescent kids.
And there's no other real integration testing happening OPENLY on the full
stack. Despite numerous claims, that is.

Software comes with bugs - this is a somewhat expected phenomena. However, bug
fixes shouldn't be mixed with new features, increasing entropy in the system.
In other words, the development process should fan-in. A process with multiple
consequent stable releases helps to achieve it; and compatibility issues would
be addressed by working on the next major release.

The model above leaves downstream with a choice of sticking to the 3.x or switching
to 4.x and so on. Where's having permanent alpha tag is a convenient way to control
software project that effectively became a vendor-controlled effort.

And yes - this leads to fragmentation, makes no mistakes about it. Because no
one can sit on the hands for a year and wait until a usable release with all
great features will come about: lot of organizations just silently forking away
to make their own environment suitable for production or sale; some of them
might sporadically contribute something back. And of course - this is not the
aim of Apache project to produce commercial grade platform.

Cos

On Wed, May 15, 2013 at 02:54PM, Roman Shaposhnik wrote:
> On Wed, May 15, 2013 at 2:14 PM, Vinod Kumar Vavilapalli
> <[EMAIL PROTECTED]> wrote:
> > Please list down all the issues that BigTop ran into *because of* new features.
>
> Whether the bug is *because of* new feature or not is a red herring
> for my argument. Please lets drop this distinction. I never used it.
>
> > You continue to argue that new features are destabilizing 2.0.*,
> > which I don't agree with at all. 2.0.3-alpha was the last time major
> > features got merged in, and we found blockers irrespective of those.
>
> This is not my argument at all. I apologize if somehow I failed to
> communicate it, but here's what my argument boils down to:
> given *my* experience with Hadoop 2.0.x series and Bigtop
> release every time I try a different release of Hadoop 2.0.x
> I run into issues that scare me. They scare me because
> they are so basic yet they make component like Sqoop
> and Oozie (and I believe Giraph on one occasion)
> pretty much DOA for YARN-base mapreduce implementation.
>
> In my mind, what that translates into is the fact that nobody
> did *any* real testing of a particular downstream component
> running on a given Hadoop 2.0.x release. Like I said --
> the issues so far make the components in question DOA.

> Effectively the onion of issues remain unpeeled, so to speak.
>
> What I'm asking on this thread (and somehow nobody is willing
> to give me a straight answer) is whether the Hadoop community
> is willing to invest in peeling this onion of issues somewhat more
> before declaring Hadoop 2.0.5 a beta release.
>
> Once again it is a binary question -- please give me an answer
> of yes or no.
>
> > I am not arguing that new features *may* destabilize the branch, but you've repeatedly stated this as if that were a fact.
>
> Your list of issues is pretty complete (give or take a few that I didn't file
> but Cos and others did). And I'd be the first one to agree that
+
Vinod Kumar Vavilapalli 2013-05-16, 06:34
+
Arun C Murthy 2013-05-15, 21:20
+
Arun C Murthy 2013-05-15, 22:02
+
Steve Loughran 2013-05-15, 22:31
+
Roman Shaposhnik 2013-05-15, 22:50
+
Arun C Murthy 2013-05-15, 23:47
+
Matt Foley 2013-05-15, 23:54
+
Konstantin Boudnik 2013-05-16, 06:19
+
Steve Loughran 2013-05-16, 17:22
+
Roman Shaposhnik 2013-05-17, 19:52
+
Andrew Purtell 2013-05-16, 04:25
+
Suresh Srinivas 2013-05-16, 05:05
+
Andrew Purtell 2013-05-16, 05:41
+
Suresh Srinivas 2013-05-16, 05:52
+
Konstantin Boudnik 2013-05-16, 06:38
+
Suresh Srinivas 2013-05-16, 07:20
+
Arun C Murthy 2013-05-16, 18:18
+
Andrew Purtell 2013-05-16, 05:29
+
Vinod Kumar Vavilapalli 2013-05-15, 21:15
+
Devaraj Das 2013-05-15, 20:39
+
Robert Evans 2013-05-16, 15:19
+
lohit 2013-05-16, 05:29
+
Doug Cutting 2013-05-17, 22:48
+
Konstantin Shvachko 2013-05-21, 08:33
+
sanjay Radia 2013-05-21, 06:27
+
Mayank Bansal 2013-05-22, 20:17