Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # dev - Kafka/Hadoop consumers and producers


Copy link to this message
-
Kafka/Hadoop consumers and producers
Jay Kreps 2013-07-03, 00:02
We currently have a contrib package for consuming and producing messages
from mapreduce (
https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=tree;f=contrib;h=e53e1fb34893e733b10ff27e79e6a1dcbb8d7ab0;hb=HEAD
).

We keep running into problems (e.g. KAFKA-946) that are basically due to
the fact that the Kafka committers don't seem to mostly be Hadoop
developers and aren't doing a good job of maintaining this code (keeping it
tested, improving it, documenting it, writing tutorials, getting it moved
over to the more modern apis, getting it working with newer Hadoop
versions, etc).

A couple of options:
1. We could try to get someone in the Kafka community (either a current
committer or not) who would adopt this as their baby (it's not much code).
2. We could just let Camus take over this functionality. They already have
a more sophisticated consumer and the producer is pretty minimal.

So are there any people who would like to adopt the current Hadoop contrib
code?

Conversely would it be possible to provide the same or similar
functionality in Camus and just delete these?

-Jay

 
+
sudheer1414.m@... 2013-10-02, 14:26
+
Cosmin Lehene 2013-07-03, 09:57
+
Felix GV 2013-07-03, 17:57
+
Ken Goodhope 2013-07-04, 04:29
+
Jay Kreps 2013-07-03, 23:49
+
aotto@... 2013-08-07, 16:42
+
Ken 2013-08-08, 04:51
+
psaltis.andrew@... 2013-08-08, 14:11
+
Felix GV 2013-08-09, 01:19
+
Andrew Psaltis 2013-08-09, 03:22
+
dibyendu.bhattachary...@... 2013-08-10, 04:02
+
Andrew Psaltis 2013-08-09, 15:52
+
Ken Goodhope 2013-08-09, 19:27
+
Jay Kreps 2013-08-10, 22:30
+
Kam Kasravi 2013-08-13, 00:17
+
Andrew Otto 2013-08-13, 02:01
+
Kam Kasravi 2013-08-13, 17:34
+
Andrew Otto 2013-08-13, 20:03
+
Kam Kasravi 2013-08-13, 22:03
+
Andrew Otto 2013-08-13, 23:17
+
Andrew Psaltis 2013-08-13, 03:20
+
Kam Kasravi 2013-08-13, 17:27
+
Andrew Otto 2013-08-13, 14:47
+
Abhi Basu 2013-11-22, 17:33