Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> [DISCUSS] Feature bloat and contrib module


+
Hari Shreedharan 2013-12-14, 00:21
Copy link to this message
-
Re: [DISCUSS] Feature bloat and contrib module
Hi Hari,
thank you very much for starting the discussion about contrib module. Flume is a very pluggable project where user have opportunity to plug in variety of different components (sink, source, channel, interceptor, ...). I believe that having a place where users can easily exchange their own plugins would be extremely useful and would help with further adoption of the project. Having a contrib module seems as one way how to accomplish this, another way would be perhaps to have a special wiki page (which we already do have [1]).

I do see benefits of repository based contrib module in having the ability to track the authorship of the changes and their history as the time passed. As I do see the contrib space more as a user driven, so I would see the bar of including a new component very low. It should be very easy for one user to submit component that works in his particular environment and let other user to tweak it for yet another purpose. With the bar keeping low we can't expect that the components will always work with current flume version nor that they will be backward compatible. Having said that I don't think that such contrib module should be official part of a Flume release. We can do our best to make a contrib release as a part of our usual release process and offer it as a separate download. I would not consider broken or removed components in the contrib as blocker for flume release though.

I'm happy to hear others opinions!

Jarcec

Links:
1: https://cwiki.apache.org/confluence/display/FLUME/Flume+NG+Plugins

On Fri, Dec 13, 2013 at 04:21:29PM -0800, Hari Shreedharan wrote:
> Hi all  
>
> Over the past few weeks, there were some discussions about including additional features in core flume itself. This led to a discussion about adding a contrib module. I thought I’d start an official discussion regarding this. The argument for contrib module was that there are too many components without generic use which are getting submitted/committed. The use-cases for such components  are limited, and hence they should not be part of core flume itself. First, we should answer the question if we want to separate components into a contrib module and why? What components go into contrib and what into core flume? What does it mean to be make a component part of the contrib module. Do contrib components get released with Flume? Can they break compatibility with older versions (what does this mean if they are not getting released?) etc. How supported are these?
>
> Please respond and let us know your view!  
>
>
> Thanks,
> Hari
>
+
Ashish 2013-12-16, 17:51
+
Israel Ekpo 2013-12-16, 20:17
+
Mike Percy 2013-12-17, 01:49
+
Ashish 2013-12-17, 05:17
+
Bruno Mahé 2013-12-17, 07:34
+
Ashish 2013-12-18, 16:46
+
Bruno Mahé 2013-12-19, 08:51
+
Mike Percy 2013-12-21, 00:01
+
Bruno Mahé 2013-12-21, 23:29
+
Gabriel Commeau 2013-12-23, 17:04
+
Ashish 2014-01-16, 08:44
+
Mike Percy 2014-01-22, 23:17
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB