Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> [DISCUSS] Flume 1.4 release plan


+
Mike Percy 2013-05-22, 07:33
+
Hari Shreedharan 2013-05-22, 07:45
+
Arvind Prabhakar 2013-05-22, 08:02
+
Jarek Jarcec Cecho 2013-05-22, 08:09
+
Alexander Alten-Lorenz 2013-05-22, 09:02
+
Ralph Goers 2013-05-26, 14:58
+
Mike Percy 2013-05-28, 13:47
+
Ralph Goers 2013-05-29, 15:49
+
Roshan Naik 2013-05-29, 22:36
+
Roshan Naik 2013-06-06, 18:29
+
Mike Percy 2013-06-12, 00:16
+
Edward Sargisson 2013-05-22, 16:28
+
Will McQueen 2013-05-22, 16:41
+
Venkatesh S R 2013-05-22, 16:51
+
Mubarak Seyed 2013-05-22, 17:41
+
Roshan Naik 2013-05-22, 18:25
+
Brock Noland 2013-05-22, 22:30
Copy link to this message
-
Re: [DISCUSS] Flume 1.4 release plan
Hi all,
I thought I'd give an update from the ElasticSearchSink perspective. I know
that we're very keen to see a 1.4 release and will test RCs, etc. when they
come out when priorities allow.

1. The patch to build with elasticsearch 0.90.1 is in FLUME-2049 and the
review is here: https://reviews.apache.org/r/11581/
I think we really need that one in Flume because otherwise anybody who
wants to use Flume with anything other than elasticsearch 0.19 (really old)
will have to do a custom compile.

2. I mentioned issues with Flume logging via itself. I got those to a point
where I thought I understood the problem and have raised work items either
in Flume or with Log4j2 to document. These do not hold up Flume 1.4.0

3. Ideally, we'd do FLUME-2092 to document that the index name needs to
match a Kibana setting.

4. It would probably be nice to get the patch from FLUME-2089 in
(unexpected character sets cause a YAMLException). However, I think we need
a unit test for this.

Cheers,
Edward

On Wed, May 22, 2013 at 9:28 AM, Edward Sargisson <[EMAIL PROTECTED]> wrote:

> Hi All,
> +1/+1 for 1.4 and Mike.
>
>
> I'm very keen to have a 1.4 for the environments I manage. There's a lot
> of stuff I'm keen on in there.
>
> On my pre-1.4 list:
> 1. compile with elasticsearch 0.90
> 2. figure out file channel state issue which is stopping Flume logging via
> itself.
>
> 1. Currently we compile with es 0.19. If somebody wants to run es 0.20
> they have to recompile (es made an interface change that is source
> compatible but requires a recompile). es 0.90 has been out for 2-ish weeks
> so safe enough to change the compile to. I think I'll raise an empty Jira
> to record this.
>
> 2. I haven't reported this because I haven't isolated it well enough. I'm
> having issues with the 1.3.1 file channel which I'd like to resolve.
>
> Cheers,
> Edward
>
> "Hi folks,
> We have had over 100 commits since 1.3.1, and a bunch of new features and
> improvements including a Thrift source, much improved ElasticSearch sink,
> support for a new plugins directory and layout, compression support in the
> avro sink/source, improved checkpointing in the file channel and more, plus
> a lot of bug fixes.
>
> It seems to me that it's time to start thinking about cutting a 1.4
> release. I would be happy to volunteer to RM the release. Worth noting that
> I will be unavailable for the next two weeks... but after that I'd be happy
> to pick this up and run with it. That's also a decent amount of time for
> people  to get moving on patches and reviews for their favorite features,
> bug fixes, etc.
>
> If this all sounds OK, I'd like to suggest targeting the last week of June
> as a release date. If we can release in time for Hadoop Summit then that
> would be pretty nice. Otherwise, if something comes up and we can't get the
> release out that week, let's shoot for the first week of July at the
> latest.
>
> Please let me know your thoughts.
>
> Regards,
> Mike
>
> +1 for Flume 1.4
> +1 for Mike being RM.
>
>
> Cheers,
> Hari"
>
+
Mike Percy 2013-06-22, 01:48
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB