Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> [DISCUSS] Flume 1.4 release plan


+
Mike Percy 2013-05-22, 07:33
+
Hari Shreedharan 2013-05-22, 07:45
+
Arvind Prabhakar 2013-05-22, 08:02
+
Jarek Jarcec Cecho 2013-05-22, 08:09
+
Alexander Alten-Lorenz 2013-05-22, 09:02
+
Ralph Goers 2013-05-26, 14:58
+
Mike Percy 2013-05-28, 13:47
+
Ralph Goers 2013-05-29, 15:49
+
Roshan Naik 2013-05-29, 22:36
+
Roshan Naik 2013-06-06, 18:29
+
Mike Percy 2013-06-12, 00:16
+
Edward Sargisson 2013-05-22, 16:28
+
Will McQueen 2013-05-22, 16:41
+
Venkatesh S R 2013-05-22, 16:51
+
Mubarak Seyed 2013-05-22, 17:41
+
Roshan Naik 2013-05-22, 18:25
+
Brock Noland 2013-05-22, 22:30
+
Edward Sargisson 2013-06-21, 21:59
Copy link to this message
-
Re: [DISCUSS] Flume 1.4 release plan
On Fri, Jun 21, 2013 at 2:59 PM, Edward Sargisson <[EMAIL PROTECTED]> wrote:

> 1. The patch to build with elasticsearch 0.90.1 is in FLUME-2049 and the
> review is here: https://reviews.apache.org/r/11581/
> I think we really need that one in Flume because otherwise anybody who
> wants to use Flume with anything other than elasticsearch 0.19 (really old)
> will have to do a custom compile.
>

I just committed this.

2. I mentioned issues with Flume logging via itself. I got those to a point
> where I thought I understood the problem and have raised work items either
> in Flume or with Log4j2 to document. These do not hold up Flume 1.4.0
>

If any of this is ready please point me to the JIRA / patch and I'll take a
look from the Flume perspective.

3. Ideally, we'd do FLUME-2092 to document that the index name needs to
> match a Kibana setting.
>

Can you provide a patch? Happy to commit it.

4. It would probably be nice to get the patch from FLUME-2089 in
> (unexpected character sets cause a YAMLException). However, I think we need
> a unit test for this.
>

If we can get a unit test, or a follow-up JIRA that includes a unit test
then I'll commit both.

BTW I'm aiming for tomorrow evening for an RC.

Thanks Edward!

Mike
On Wed, May 22, 2013 at 9:28 AM, Edward Sargisson <[EMAIL PROTECTED]> wrote:
>
>> Hi All,
>> +1/+1 for 1.4 and Mike.
>>
>>
>> I'm very keen to have a 1.4 for the environments I manage. There's a lot
>> of stuff I'm keen on in there.
>>
>> On my pre-1.4 list:
>> 1. compile with elasticsearch 0.90
>> 2. figure out file channel state issue which is stopping Flume logging
>> via itself.
>>
>> 1. Currently we compile with es 0.19. If somebody wants to run es 0.20
>> they have to recompile (es made an interface change that is source
>> compatible but requires a recompile). es 0.90 has been out for 2-ish weeks
>> so safe enough to change the compile to. I think I'll raise an empty Jira
>> to record this.
>>
>> 2. I haven't reported this because I haven't isolated it well enough. I'm
>> having issues with the 1.3.1 file channel which I'd like to resolve.
>>
>> Cheers,
>> Edward
>>
>> "Hi folks,
>> We have had over 100 commits since 1.3.1, and a bunch of new features and
>> improvements including a Thrift source, much improved ElasticSearch sink,
>> support for a new plugins directory and layout, compression support in the
>> avro sink/source, improved checkpointing in the file channel and more,
>> plus
>> a lot of bug fixes.
>>
>> It seems to me that it's time to start thinking about cutting a 1.4
>> release. I would be happy to volunteer to RM the release. Worth noting
>> that
>> I will be unavailable for the next two weeks... but after that I'd be
>> happy
>> to pick this up and run with it. That's also a decent amount of time for
>> people  to get moving on patches and reviews for their favorite features,
>> bug fixes, etc.
>>
>> If this all sounds OK, I'd like to suggest targeting the last week of June
>> as a release date. If we can release in time for Hadoop Summit then that
>> would be pretty nice. Otherwise, if something comes up and we can't get
>> the
>> release out that week, let's shoot for the first week of July at the
>> latest.
>>
>> Please let me know your thoughts.
>>
>> Regards,
>> Mike
>>
>> +1 for Flume 1.4
>> +1 for Mike being RM.
>>
>>
>> Cheers,
>> Hari"
>>
>
>