Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> [DISCUSS] Feature bloat and contrib module

Copy link to this message
Re: [DISCUSS] Feature bloat and contrib module
Hi Hari,
thank you very much for starting the discussion about contrib module. Flume is a very pluggable project where user have opportunity to plug in variety of different components (sink, source, channel, interceptor, ...). I believe that having a place where users can easily exchange their own plugins would be extremely useful and would help with further adoption of the project. Having a contrib module seems as one way how to accomplish this, another way would be perhaps to have a special wiki page (which we already do have [1]).

I do see benefits of repository based contrib module in having the ability to track the authorship of the changes and their history as the time passed. As I do see the contrib space more as a user driven, so I would see the bar of including a new component very low. It should be very easy for one user to submit component that works in his particular environment and let other user to tweak it for yet another purpose. With the bar keeping low we can't expect that the components will always work with current flume version nor that they will be backward compatible. Having said that I don't think that such contrib module should be official part of a Flume release. We can do our best to make a contrib release as a part of our usual release process and offer it as a separate download. I would not consider broken or removed components in the contrib as blocker for flume release though.

I'm happy to hear others opinions!


1: https://cwiki.apache.org/confluence/display/FLUME/Flume+NG+Plugins

On Fri, Dec 13, 2013 at 04:21:29PM -0800, Hari Shreedharan wrote:
> Hi all  
> Over the past few weeks, there were some discussions about including additional features in core flume itself. This led to a discussion about adding a contrib module. I thought I’d start an official discussion regarding this. The argument for contrib module was that there are too many components without generic use which are getting submitted/committed. The use-cases for such components  are limited, and hence they should not be part of core flume itself. First, we should answer the question if we want to separate components into a contrib module and why? What components go into contrib and what into core flume? What does it mean to be make a component part of the contrib module. Do contrib components get released with Flume? Can they break compatibility with older versions (what does this mean if they are not getting released?) etc. How supported are these?
> Please respond and let us know your view!  
> Thanks,
> Hari