Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Help with kafka-hadoop loader pipeline ???


Copy link to this message
-
Help with kafka-hadoop loader pipeline ???
Hi

We were trying to use the kafka-hadoop loader for loading messages from
kafka into the hadoop ecosystem
https://github.com/michal-harish/kafka-hadoop-loader

Our consumer is hadoop in this case. The code runs saying job successfull
and a _SUCESS file is created in hdfs.

But no output file is created in the given hdfs path, we tried sending both
json as well as protobuf files through kafka.

Please help if anyone has any idea on loading protobuf files into hadoop.
--
Chitra Raveendran
Data Scientist
*Flutura Decision Sciences & Analytics*

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB