Re: Kafka/Hadoop consumers and producers
I would like to do this refactoring since I did a high level consumer a while ago.
A few weeks ago I had opened KAFKA-949 Kafka on Yarn which I was also hoping to add to contribute.
It's almost done. KAFKA-949 is paired with BIGTOP-989 which adds kafka 0.8 to the bigtop distribution.
KAFKA-949 basically allows kafka brokers to be started up using sysvinit services and would ease some of the
startup/configuration issues that newbies have when getting started with kafka. Ideally I would like to
fold a number of kafka/bin/* commands into the kafka service. Andrew please let me know if would like to
pick this up instead. Thanks!
From: Jay Kreps <[EMAIL PROTECTED]>
To: Ken Goodhope <[EMAIL PROTECTED]>
Cc: Andrew Psaltis <[EMAIL PROTECTED]>; [EMAIL PROTECTED]; "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; Felix GV <[EMAIL PROTECTED]>; Cosmin Lehene <[EMAIL PROTECTED]>; "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Sent: Saturday, August 10, 2013 3:30 PM
Subject: Re: Kafka/Hadoop consumers and producers
So guys, just to throw my 2 cents in:
1. We aren't deprecating anything. I just noticed that the Hadoop contrib
package wasn't getting as much attention as it should.
2. Andrew or anyone--if there is anyone using the contrib package who would
be willing to volunteer to kind of adopt it that would be great. I am happy
to help in whatever way I can. The practical issue is that most of the
committers are either using Camus or not using Hadoop at all so we just
haven't been doing a good job of documenting, bug fixing, and supporting
the contrib packages.
3. Ken, if you could document how to use Camus that would likely make it a
lot more useful to people. I think most people would want a full-fledged
ETL solution and would likely prefer Camus, but very few people are using
On Fri, Aug 9, 2013 at 12:27 PM, Ken Goodhope <[EMAIL PROTECTED]> wrote: