Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> [DISCUSS] Move common, hdfs, mapreduce contrib components to apache-extras.org or elsewhere


Copy link to this message
-
Re: [DISCUSS] Move common, hdfs, mapreduce contrib components to apache-extras.org or elsewhere
+1 to this process in general.

In particular, tools like MRUnit can benefit from having an independent
release due to where they are used in a project's lifecycle. MRUnit should
be specified as a test dependency, whereas Hadoop itself is a
compile/runtime dependency.  As it stands, there isn't an easy way to manage
this. This would increase flexibility for this tool, probably for others as
well.

- Aaron

On Mon, Jan 31, 2011 at 3:23 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:

> On Sun, Jan 30, 2011 at 11:19 PM, Owen O'Malley <[EMAIL PROTECTED]>
> wrote:
>
> >
> > Also note that pushing code out of Hadoop has a high cost. There are at
> > least 3 forks of the hadoop-gpl-compression code. That creates a lot of
> > confusion for the users. A lot of users never go to the work to figure
> out
> > which fork and branch of hadoop-gpl-compression work with the version of
> > Hadoop they installed.
> >
> >
> Indeed it creates confusion, but in my opinion it has been very successful
> modulo that confusion.
>
> In particular, Kevin and I (who each have a repo on github but basically
> co-maintain a branch) have done about 8 bugfix releases of LZO in the last
> year. The ability to take a bug and turn it around into a release within a
> few days has been very beneficial to the users. If it were part of core
> Hadoop, people would be forced to live with these blocker bugs for months
> at
> a time between dot releases.
>
> IMO the more we can take non-core components and move them to separate
> release timelines, the better. Yes, it is harder for users, but it also is
> easier for them when they hit a bug - they don't have to wait months for a
> wholesale upgrade which might contain hundreds of other changes to core
> components. I think this will also help the situation where people have set
> up shop on branches -- a lot of the value of these branches comes from the
> frequency of backports and bugfixes to "non-core" components. If the
> non-core stuff were on a faster timeline upstream, we could maintain core
> stability while also offering people the latest and greatest libraries,
> tools, codecs, etc.
>
> -Todd
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB