Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Help with kafka-hadoop loader pipeline ???


Copy link to this message
-
Re: Help with kafka-hadoop loader pipeline ???
Jun Rao 2013-06-04, 14:54
I am not familiar with the hadoop loader that you are using. At LinkedIn,
we are using https://github.com/linkedin/camus. It has an 0.8
implementation too: https://github.com/linkedin/camus/tree/camus-kafka-0.8/

Thanks,

Jun
On Tue, Jun 4, 2013 at 3:32 AM, Chitra Raveendran <
[EMAIL PROTECTED]> wrote:

> Hi
>
> We were trying to use the kafka-hadoop loader for loading messages from
> kafka into the hadoop ecosystem
> https://github.com/michal-harish/kafka-hadoop-loader
>
> Our consumer is hadoop in this case. The code runs saying job successfull
> and a _SUCESS file is created in hdfs.
>
> But no output file is created in the given hdfs path, we tried sending both
> json as well as protobuf files through kafka.
>
> Please help if anyone has any idea on loading protobuf files into hadoop.
>
>
> --
> Chitra Raveendran
> Data Scientist
> *Flutura Decision Sciences & Analytics*
>