Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> ETL with Kafka


+
Guy Doulberg 2013-01-06, 07:49
+
David Arthur 2013-01-06, 22:29
Copy link to this message
-
Re: ETL with Kafka
You can use Kafka to store data on Hadoop via the Hadoop consumer in
contrib, and then use Talend or Pig to ETL it, before finally emitting the
ETL's records via the Hadoop producer in contrib.

https://github.com/kafka-dev/kafka/tree/master/contrib
http://docs.hortonworks.com/CURRENT/index.htm#Data_Integration_Services_With_HDP/Using_Data_Integration_Services_Powered_By_Talend/Using_Talend.htm

Russell Jurney http://datasyndrome.com

On Jan 6, 2013, at 2:29 PM, David Arthur <[EMAIL PROTECTED]> wrote:

Storm has support for Kafka, if that's the sort of thing you're looking
for. Maybe you could describe your use case a bit more?

On Sunday, January 6, 2013, Guy Doulberg wrote:

Hi
I am looking for an ETL tool that can connect to kafka, as a consumer and

as a producer,
Have you heard of such a tool?
Thanks

Guy
--
David Arthur

 
+
Guy Doulberg 2013-01-07, 07:12
+
Ken Krugler 2013-01-07, 17:57
+
Russell Jurney 2013-01-07, 20:48
+
Ken Krugler 2013-01-07, 21:51
+
Russell Jurney 2013-01-07, 22:06
+
Ken Krugler 2013-01-07, 22:21
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB