Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Help with kafka-hadoop loader pipeline ???


Copy link to this message
-
Re: Help with kafka-hadoop loader pipeline ???
I am not familiar with the hadoop loader that you are using. At LinkedIn,
we are using https://github.com/linkedin/camus. It has an 0.8
implementation too: https://github.com/linkedin/camus/tree/camus-kafka-0.8/

Thanks,

Jun
On Tue, Jun 4, 2013 at 3:32 AM, Chitra Raveendran <
[EMAIL PROTECTED]> wrote:

> Hi
>
> We were trying to use the kafka-hadoop loader for loading messages from
> kafka into the hadoop ecosystem
> https://github.com/michal-harish/kafka-hadoop-loader
>
> Our consumer is hadoop in this case. The code runs saying job successfull
> and a _SUCESS file is created in hdfs.
>
> But no output file is created in the given hdfs path, we tried sending both
> json as well as protobuf files through kafka.
>
> Please help if anyone has any idea on loading protobuf files into hadoop.
>
>
> --
> Chitra Raveendran
> Data Scientist
> *Flutura Decision Sciences & Analytics*
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB