Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> Re: [VOTE] - Release 2.0.5-beta

Copy link to this message
Re: [VOTE] - Release 2.0.5-beta
The other thread or "vote" or whatever at least served the purpose in fresh
surfacing of concerns. Talk of new features going in to a "beta" on a very
short short timetable is concerning for anyone with experience working on
large software projects. It's not a little ironic that this vote thread,
done in response to sort out the other one predicated on stability
concerns, begins with a laundry list of features and JIRAs to go in. I
think it is usually the case that a beta release receives only bugfixes*
over the alpha that proceeded it. This may just be a lack of consensus on
what "beta" means.

Please set aside discussion on particular features or Hadoop bylaws or
politics or debate club. I can't speak for all of downstream of course, but
to the extent that I can I can say we don't care about that. The core ask,
at least mine, is take a fresh look at reducing per-release disruptions to
the rest of the entire ecosystem that has grown up around Hadoop. Any
change made in Hadoop does not happen in isolation. On that other thread
were good suggestions on more frequent releases, reducing the change scope
per release, adopting merge windows, etc., all with the aim of allowing
downstream the time to sort out integration issues. The other comment on
this thread that suggests ASF governance structures being inadequate for
negotiating changes in a large ecosystem might be on to something, but at
the same time Apache BigTop may be an effective ASF-native answer to that.
Time will tell. It seems the tone of discussions between BigTop and Hadoop
in this thread are better than in the past. I will take further discussion
on volunteering resources to the BigTop lists.

* - I think this would include the great recent work in YARN on API
stability and in MR on restoring some backwards compatibility with MR1, but
perhaps these only meet my personal definition of bugfix, granted.

On Thu, May 16, 2013 at 5:20 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> Great summary, thanks Vinod.
> On May 15, 2013, at 2:14 PM, Vinod Kumar Vavilapalli wrote:
> >
> > Roman, I keep this same argument again and again. Should've refuted
> earlier.
> >
> > Please list down all the issues that BigTop ran into *because of* new
> features. You continue to argue that new features are destabilizing 2.0.*,
> which I don't agree with at all. 2.0.3-alpha was the last time major
> features got merged in, and we found blockers irrespective of those.
> >
> > MAPREDUCE-5240 specifically isn't due to any feature merge. This was a
> bug. I'd say this is a long standing bug in 2.0.x. You sure this passed in
> 2.0.3? Even so, this is mostly broken by another bug-fix and *not* because
> of any feature.
> >
> > I quickly checked other bugs you reported in 2.0.x:
> > - MAPREDUCE-5088 was caused by the fix for HADOOP-9299 which was again a
> long standing issue in 2.0.x
> > - MAPREDUCE-3728 is similar
> > - MAPREDUCE-5117 is similar
> > - MAPREDUCE-4219 was a security related feature request from you.
> > - MAPREDUCE-3916 was because of new proxy-server added.
> >
> > I am not arguing that new features *may* destabilize the branch, but
> you've repeatedly stated this as if that were a fact.
> >
> > Really appreciate the testing done by BigTop, but please don't distort
> the facts.
> >
> > Thanks,
> > +Vinod
> >
> >
> > On May 15, 2013, at 1:29 PM, Roman Shaposhnik wrote:
> >
> >> Please tell me if my expectations are incorrect, but to me the -beta
> would
> >> signify it being a 'safe' target for the downstream components. We're
> still
> >> finding *very* basic and *very* disruptive issues (MAPREDUCE-5240 is
> >> a good example) that essentially mean DOA for downstream that depends
> >> on this functionality.
> >>
> >> Are we comfortable with delivering 2.0.5-beta and later on starting
> >> to discover things like MAPREDUCE-5240 more or less accidentally?
> >
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)