Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Kafka Hadoop Consumer for multiple brokers

Copy link to this message
Re: Kafka Hadoop Consumer for multiple brokers
The idea is that each mapper is only connecting to a single Kafka broker.
Each line in the input file specifies broker uri, topic, partition and

The hadoop consumer in contrib is probably a bit outdated. The one that
LinkedIn uses now can be found at https://github.com/linkedin/camus


On Tue, Jun 4, 2013 at 7:29 AM, Samir Madhavan <

> Hi,
> I was going through the hadoop-consumer in the contrib folder. There is
> property that asks for the kafka server URI. This might sound silly but
> from looking at it, it seems to be only for a single kafka broker.
> What we have multiple brokers, how do we implement the hadoop-consumer for
> it?
> Regards,
> Samir