Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Bigtop >> mail # dev >> [DISCUSS] stabilizing Hadoop releases wrt. downstream


Copy link to this message
-
Re: [DISCUSS] stabilizing Hadoop releases wrt. downstream
Thanks Bobby. I 've setup a unit test
job<https://builds.apache.org/view/Hadoop/job/Hadoop-Common-2-Build/>to
execute unit test's on branch-2 on a daily basis.
I'm happy to help with build setup. Let me know.

-Giri
On Tue, Mar 5, 2013 at 9:02 PM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:

> Great start, Bobby! I certainly can jump on fix something quickly if
> needed as
> well (neither an RE person, but CI is truly a dev. tool!)
>
> Thanks!
>   Cos
>
> On Tue, Mar 05, 2013 at 07:18AM, Robert Evans wrote:
> > That is a great point.  I have been meaning to set up the Jenkins build
> > for branch-2 for a while, so I took the 10 mins and just did it.
> >
> > https://builds.apache.org/job/Hadoop-Common-2-Commit/
> >
> > Don't let the name fool you, it publishes not just common, but HDFS,
> YARN,
> > MR, and tools too.  You should now have branch-2 SNAPSHOTS updated on
> each
> > commit to branch-2.  Feel free to bug me if you need more integration
> > points.  I am not an RE guy, but I can hack it to make things work :)
> >
> > --Bobby
> >
> > On 3/5/13 12:15 AM, "Konstantin Boudnik" <[EMAIL PROTECTED]> wrote:
> >
> > >Arun,
> > >
> > >first of all, I don't think anyone is trying to put a blame on someone
> > >else. E.g. I had similar experience with Oozie being broken because of
> > >certain released changes in the upstream.
> > >
> > >I am sure that most people in BigTop community - especially those who
> > >share the committer-ship privilege in BigTop and other upstream
> > >projects, including Hadoop, - would be happy to help with the
> > >stabilization of the Hadoop base. The issue that a downstream
> > >integration project is likely to have is - for once - the absence of
> > >regularly published development artifacts. In the light of "it didn't
> > >happen if there's no picture" here's a couple of examples:
> > >
> > >  - 2.0.2-SNAPSHOT weren't published at all; only release 2.0.2-alpha
> > >artifacts were
> > >  - 2.0.3-SNAPSHOT weren't published until Feb 29, 2013 (it happened
> just
> > >once)
> > >
> > >So, technically speaking, unless an integration project is willing to
> > >build and maintain its own artifacts, it is impossible to do any
> > >preventive validation.
> > >
> > >Which brings me to my next question: how do you guys address
> > >"Integration is high on the list of *every* release". Again, please
> > >don't get me wrong - I am not looking to lay a blame on or corner
> > >anyone - I am really curious and would appreciate the input.
> > >
> > >
> > >Vinod:
> > >
> > >> As you yourself noted later, the pain is part of the 'alpha' status
> > >> of the release. We are targeting +one of the immediate future
> > >> releases to be a beta and so these troubles are really only the
> > >> short +term.
> > >
> > >I don't really want to get into the discussion about of what
> > >constitutes the alpha and how it has delayed the adoption of Hadoop2
> > >line. However, I want to point out that it is especially important for
> > >"alpha" platform to work nicely with downstream consumers of the said
> > >platform. For quite obvious reasons, I believe.
> > >
> > >> I think there is a fundamental problem with the interaction of
> > >> Bigtop with the downstream projects, if nothing else, with
> > >
> > >BigTop is as downstream as it can get, because BigTop essentially
> > >consumes all other component releases in order to produce a viable
> > >stack. Technicalities aside...
> > >
> > >> Hadoop. We never formalized on the process, will BigTop step in
> > >> after an RC is up for vote or before? As I see it, it's happening
> > >
> > >Bigtop essentially can give any component, including Hadoop, and
> > >better yet - the set of components - certain guaratees about
> > >compatibility and dependencies being included. Case in point is
> > >missing commons libraries missed in 1.0.1 release that essentially
> > >prevented HBase from working properly.
> > >
> > >> after the vote is up, so no wonder we are in this state. Shall we
> > >> have a pre-notice to Bigtop so that it can step in before?
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB