Kafka, mail # user - Kafka/Hadoop consumers and producers - 2013-07-03, 00:02
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Plain View
Felix GV 2013-07-04, 17:48
Copy link to this message
Kafka/Hadoop consumers and producers
We currently have a contrib package for consuming and producing messages
from mapreduce (

We keep running into problems (e.g. KAFKA-946) that are basically due to
the fact that the Kafka committers don't seem to mostly be Hadoop
developers and aren't doing a good job of maintaining this code (keeping it
tested, improving it, documenting it, writing tutorials, getting it moved
over to the more modern apis, getting it working with newer Hadoop
versions, etc).

A couple of options:
1. We could try to get someone in the Kafka community (either a current
committer or not) who would adopt this as their baby (it's not much code).
2. We could just let Camus take over this functionality. They already have
a more sophisticated consumer and the producer is pretty minimal.

So are there any people who would like to adopt the current Hadoop contrib

Conversely would it be possible to provide the same or similar
functionality in Camus and just delete these?


Sam Meder 2013-07-03, 08:08
Vadim Keylis 2013-07-03, 08:20
Cosmin Lehene 2013-07-03, 09:57
Felix GV 2013-07-03, 17:57
Jay Kreps 2013-07-03, 23:49
Ken Goodhope 2013-07-04, 04:29
aotto@... 2013-08-07, 16:42
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB