Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Re: Kafka/Hadoop consumers and producers


Copy link to this message
-
Re: Kafka/Hadoop consumers and producers
I am also interested with hadoop+kafka capabilities. I am using kafka 0.7 ,
so my question : What is the best way to consume contect from kafka and
write it to hdfs? At this time I need the only consuming functionality.

thanks
Oleg.
On Wed, Aug 7, 2013 at 7:33 PM, <[EMAIL PROTECTED]> wrote:

> Hi all,
>
> Over at the Wikimedia Foundation, we're trying to figure out the best way
> to do our ETL from Kafka into Hadoop.  We don't currently use Avro and I'm
> not sure if we are going to.  I came across this post.
>
> If the plan is to remove the hadoop-consumer from Kafka contrib, do you
> think we should not consider it as one of our viable options?
>
> Thanks!
> -Andrew

 
+
Oleg Ruchovets 2013-08-09, 17:27
+
Felix GV 2013-08-09, 18:07
+
Andrew Otto 2013-08-09, 18:49
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB