Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> [DISCUSS] Flume 1.4 release plan

Copy link to this message
Re: [DISCUSS] Flume 1.4 release plan
On Fri, Jun 21, 2013 at 2:59 PM, Edward Sargisson <[EMAIL PROTECTED]> wrote:

> 1. The patch to build with elasticsearch 0.90.1 is in FLUME-2049 and the
> review is here: https://reviews.apache.org/r/11581/
> I think we really need that one in Flume because otherwise anybody who
> wants to use Flume with anything other than elasticsearch 0.19 (really old)
> will have to do a custom compile.

I just committed this.

2. I mentioned issues with Flume logging via itself. I got those to a point
> where I thought I understood the problem and have raised work items either
> in Flume or with Log4j2 to document. These do not hold up Flume 1.4.0

If any of this is ready please point me to the JIRA / patch and I'll take a
look from the Flume perspective.

3. Ideally, we'd do FLUME-2092 to document that the index name needs to
> match a Kibana setting.

Can you provide a patch? Happy to commit it.

4. It would probably be nice to get the patch from FLUME-2089 in
> (unexpected character sets cause a YAMLException). However, I think we need
> a unit test for this.

If we can get a unit test, or a follow-up JIRA that includes a unit test
then I'll commit both.

BTW I'm aiming for tomorrow evening for an RC.

Thanks Edward!

On Wed, May 22, 2013 at 9:28 AM, Edward Sargisson <[EMAIL PROTECTED]> wrote:
>> Hi All,
>> +1/+1 for 1.4 and Mike.
>> I'm very keen to have a 1.4 for the environments I manage. There's a lot
>> of stuff I'm keen on in there.
>> On my pre-1.4 list:
>> 1. compile with elasticsearch 0.90
>> 2. figure out file channel state issue which is stopping Flume logging
>> via itself.
>> 1. Currently we compile with es 0.19. If somebody wants to run es 0.20
>> they have to recompile (es made an interface change that is source
>> compatible but requires a recompile). es 0.90 has been out for 2-ish weeks
>> so safe enough to change the compile to. I think I'll raise an empty Jira
>> to record this.
>> 2. I haven't reported this because I haven't isolated it well enough. I'm
>> having issues with the 1.3.1 file channel which I'd like to resolve.
>> Cheers,
>> Edward
>> "Hi folks,
>> We have had over 100 commits since 1.3.1, and a bunch of new features and
>> improvements including a Thrift source, much improved ElasticSearch sink,
>> support for a new plugins directory and layout, compression support in the
>> avro sink/source, improved checkpointing in the file channel and more,
>> plus
>> a lot of bug fixes.
>> It seems to me that it's time to start thinking about cutting a 1.4
>> release. I would be happy to volunteer to RM the release. Worth noting
>> that
>> I will be unavailable for the next two weeks... but after that I'd be
>> happy
>> to pick this up and run with it. That's also a decent amount of time for
>> people  to get moving on patches and reviews for their favorite features,
>> bug fixes, etc.
>> If this all sounds OK, I'd like to suggest targeting the last week of June
>> as a release date. If we can release in time for Hadoop Summit then that
>> would be pretty nice. Otherwise, if something comes up and we can't get
>> the
>> release out that week, let's shoot for the first week of July at the
>> latest.
>> Please let me know your thoughts.
>> Regards,
>> Mike
>> +1 for Flume 1.4
>> +1 for Mike being RM.
>> Cheers,
>> Hari"