Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Node-Kafka Client Review and Question


Copy link to this message
-
Node-Kafka Client Review and Question
Hi Everyone,

I just wanted to follow-up on a previous thread concerning our investigation of identifying a stable Node-Kafka client. To date we have tested the following:

1. Franz-Kafka (https://github.com/dannycoates/franz-kafka)
2. Node-Kafka (v2.1, https://github.com/radekg/node-kafka)
3. Node-Kafka (v2.3, https://github.com/marcuswestin/node-kafka)
4. Prozess (v0.3.5, https://github.com/cainus/Prozess)

Results:

1. Could not get Franz-Kafka and Prozess to work. Requires funky dependencies.
2. Node-Kafka, v2.1 was successfully setup but performed less stable than #3.
3. Node-Kafka, v2.3 was successfully setup, exhibited the best performance profile but the consumer is highly inconsistent - specifically, consumer object remained in-memory regardless what we did (i.e. var consumer = undefined after receiving message). Nothing appears to mitigate this and ALL consumed messaged get replayed on reception of a new message.

With this said, is there a Node-Kafka client people are actually using in production that doesn't exhibit the profiles we have seen? We have back-tracked using Node-Kafka (v2.3) to only produce messages and rely on Redis PubSub channels for asynchronous acking of these messages. We would be willing to roll-up our sleeves with the community to develop a much more stable Node-Kafka client.

Kind regards,

Chris Alexander
Chief Technical Architect and Engineer
Gravy, Inc.

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB