Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Re: Kafka/Hadoop consumers and producers


Copy link to this message
-
Re: Kafka/Hadoop consumers and producers
Oleg Ruchovets 2013-08-07, 19:44
I am also interested with hadoop+kafka capabilities. I am using kafka 0.7 ,
so my question : What is the best way to consume contect from kafka and
write it to hdfs? At this time I need the only consuming functionality.

thanks
Oleg.
On Wed, Aug 7, 2013 at 7:33 PM, <[EMAIL PROTECTED]> wrote:

> Hi all,
>
> Over at the Wikimedia Foundation, we're trying to figure out the best way
> to do our ETL from Kafka into Hadoop.  We don't currently use Avro and I'm
> not sure if we are going to.  I came across this post.
>
> If the plan is to remove the hadoop-consumer from Kafka contrib, do you
> think we should not consider it as one of our viable options?
>
> Thanks!
> -Andrew

 
+
Oleg Ruchovets 2013-08-09, 17:27
+
Felix GV 2013-08-09, 18:07
+
Andrew Otto 2013-08-09, 18:49