Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> A (nginx_module/Perl) Producer that writes to a Kafka Cluster with Replication

Tobias Feldhaus 2012-12-19, 10:03
Jun Rao 2012-12-19, 15:47
Copy link to this message
Re: A (nginx_module/Perl) Producer that writes to a Kafka Cluster with Replication
Hi  Tobias

I am not using a nginx but a tomcat server, but I solved the same
problem in this tomcat server.

I created a Logback appender that writes to Kafka,

My tomcat server uses logback to log each of the requests,

I configured this logback to write to a file, and to write to a kafka
cluster (using the mentioned appender)

If you like, I can try to share my code,
On 12/19/2012 12:02 PM, Tobias Feldhaus wrote:
> Hi,
> I am searching for a Kafka Producer that is able to write to a Kafka Cluster with replication, so that there is no data lost when one broker is going down. This functionality has been added in Kafka 0.8 as far as I know. In the current (0.6) configuration we setup the Producer as a Perl Module inside a nginx web server to handle HTTP requests. I am trying to achieve a similar functionality, but with a Kafka-Cluster/ZooKeeper combination instead of a single Kafka server behind it.
> What would be the best way to achieve this? I haven't found a Perl Module that can talk to a Kafka/ZooKeeper configuration so far.
> Regards,
> Tobias