Kafka, mail # user - Re: ETL with Kafka - 2013-01-07, 17:57
 Search Hadoop and all its subprojects:
Copy link to this message
-
Re: ETL with Kafka
Hi Guy,

On Jan 6, 2013, at 11:11pm, Guy Doulberg wrote:
Interesting - we build ETLs on top of Hadoop using Cascading (open source workflow API), which has a lot of what it calls "Taps" for connecting to data sources and sinks.

But I haven't heard of a Kafka Tap. Should be possible to implement, though.

One issue is that Hadoop is batch oriented, so there's a bit of an impedance mismatch when you've got a streaming data source, but from experience it's possible to get that to work.

Ken Krugler
+1 530-210-6378
http://www.scaleunlimited.com
custom big data solutions & training
Hadoop, Cascading, Cassandra & Solr
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB