Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> [DISCUSS] Flume 1.4 release plan


+
Mike Percy 2013-05-22, 07:33
+
Hari Shreedharan 2013-05-22, 07:45
+
Arvind Prabhakar 2013-05-22, 08:02
+
Jarek Jarcec Cecho 2013-05-22, 08:09
+
Alexander Alten-Lorenz 2013-05-22, 09:02
+
Ralph Goers 2013-05-26, 14:58
+
Mike Percy 2013-05-28, 13:47
+
Ralph Goers 2013-05-29, 15:49
+
Roshan Naik 2013-05-29, 22:36
+
Roshan Naik 2013-06-06, 18:29
+
Mike Percy 2013-06-12, 00:16
+
Edward Sargisson 2013-05-22, 16:28
+
Will McQueen 2013-05-22, 16:41
+
Venkatesh S R 2013-05-22, 16:51
+
Mubarak Seyed 2013-05-22, 17:41
+
Roshan Naik 2013-05-22, 18:25
+
Brock Noland 2013-05-22, 22:30
Copy link to this message
-
Re: [DISCUSS] Flume 1.4 release plan
Hi all,
I thought I'd give an update from the ElasticSearchSink perspective. I know
that we're very keen to see a 1.4 release and will test RCs, etc. when they
come out when priorities allow.

1. The patch to build with elasticsearch 0.90.1 is in FLUME-2049 and the
review is here: https://reviews.apache.org/r/11581/
I think we really need that one in Flume because otherwise anybody who
wants to use Flume with anything other than elasticsearch 0.19 (really old)
will have to do a custom compile.

2. I mentioned issues with Flume logging via itself. I got those to a point
where I thought I understood the problem and have raised work items either
in Flume or with Log4j2 to document. These do not hold up Flume 1.4.0

3. Ideally, we'd do FLUME-2092 to document that the index name needs to
match a Kibana setting.

4. It would probably be nice to get the patch from FLUME-2089 in
(unexpected character sets cause a YAMLException). However, I think we need
a unit test for this.

Cheers,
Edward

On Wed, May 22, 2013 at 9:28 AM, Edward Sargisson <[EMAIL PROTECTED]> wrote:

> Hi All,
> +1/+1 for 1.4 and Mike.
>
>
> I'm very keen to have a 1.4 for the environments I manage. There's a lot
> of stuff I'm keen on in there.
>
> On my pre-1.4 list:
> 1. compile with elasticsearch 0.90
> 2. figure out file channel state issue which is stopping Flume logging via
> itself.
>
> 1. Currently we compile with es 0.19. If somebody wants to run es 0.20
> they have to recompile (es made an interface change that is source
> compatible but requires a recompile). es 0.90 has been out for 2-ish weeks
> so safe enough to change the compile to. I think I'll raise an empty Jira
> to record this.
>
> 2. I haven't reported this because I haven't isolated it well enough. I'm
> having issues with the 1.3.1 file channel which I'd like to resolve.
>
> Cheers,
> Edward
>
> "Hi folks,
> We have had over 100 commits since 1.3.1, and a bunch of new features and
> improvements including a Thrift source, much improved ElasticSearch sink,
> support for a new plugins directory and layout, compression support in the
> avro sink/source, improved checkpointing in the file channel and more, plus
> a lot of bug fixes.
>
> It seems to me that it's time to start thinking about cutting a 1.4
> release. I would be happy to volunteer to RM the release. Worth noting that
> I will be unavailable for the next two weeks... but after that I'd be happy
> to pick this up and run with it. That's also a decent amount of time for
> people  to get moving on patches and reviews for their favorite features,
> bug fixes, etc.
>
> If this all sounds OK, I'd like to suggest targeting the last week of June
> as a release date. If we can release in time for Hadoop Summit then that
> would be pretty nice. Otherwise, if something comes up and we can't get the
> release out that week, let's shoot for the first week of July at the
> latest.
>
> Please let me know your thoughts.
>
> Regards,
> Mike
>
> +1 for Flume 1.4
> +1 for Mike being RM.
>
>
> Cheers,
> Hari"
>
+
Mike Percy 2013-06-22, 01:48