Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Node-Kafka Client Review and Question


Copy link to this message
-
Re: Node-Kafka Client Review and Question
We did an examination of the tagged branch but the version was 0.1.7 and its been static for over a year now. I will say that the Node-Kafka (v2.3) producer has been stable however. A previous thread concerning Node-Kafka client development revealed that a C library will be out for 0.8, supporting more stable library client development (hint hint too ;-)

Unfortunately, we are moving rapidly with what works now to get our platform to an MVP state. Hopefully, there won't be too much refactoring when the next gen Node-Kafka client is complete. :(

Chris

----- Original Message -----
From: "Taylor Gautier" <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Wednesday, April 24, 2013 4:39:24 PM
Subject: Re: Node-Kafka Client Review and Question

This implementation is what I worked on while at Tagged, which was forked
from Marcus' version, but I don't think it ever merged back to Marcus':

https://github.com/tagged/node-kafka

It was in production for about a year when I left Tagged about 6 months
ago.  I know that there were some internal fixes that never made it back
out.  I think the version that is public is pretty stable, but it's hard
for me to know since I am no longer at Tagged and don't have access to the
internal repos to know for sure what fixes are still private and which have
been published back.

That said, I think you should definitely give this version a shot first
before moving on.

Finally, if I were to do it all over again, with about a solid additional
year of node programming experience under my belt, I'd probably rewrite
everything from the interfaces to the implementation.

In my current job there's a strong possibility of us adopting a Node.js +
Kafka implementation, but not for at least a few months, so I wouldn't
expect to be back in the community to work on this for a little while.
 Also, I'm kind of waiting for 0.8 (hint hint ;-)

On Wed, Apr 24, 2013 at 10:45 AM, Christian Carollo <[EMAIL PROTECTED]>wrote:

> Hi Everyone,
>
> I have been experimenting with the libraries listed below and experienced
> the same problems.
> I have not found any another other node clients.  I am interested in
> finding a node solution as well.
> Happy to contribute on a common solution.
>
> Christian Carollo
>
> On Apr 24, 2013, at 10:19 AM, Christopher Alexander <
> [EMAIL PROTECTED]> wrote:
>
> > Hi Everyone,
> >
> > I just wanted to follow-up on a previous thread concerning our
> investigation of identifying a stable Node-Kafka client. To date we have
> tested the following:
> >
> > 1. Franz-Kafka (https://github.com/dannycoates/franz-kafka)
> > 2. Node-Kafka (v2.1, https://github.com/radekg/node-kafka)
> > 3. Node-Kafka (v2.3, https://github.com/marcuswestin/node-kafka)
> > 4. Prozess (v0.3.5, https://github.com/cainus/Prozess)
> >
> > Results:
> >
> > 1. Could not get Franz-Kafka and Prozess to work. Requires funky
> dependencies.
> > 2. Node-Kafka, v2.1 was successfully setup but performed less stable
> than #3.
> > 3. Node-Kafka, v2.3 was successfully setup, exhibited the best
> performance profile but the consumer is highly inconsistent - specifically,
> consumer object remained in-memory regardless what we did (i.e. var
> consumer = undefined after receiving message). Nothing appears to mitigate
> this and ALL consumed messaged get replayed on reception of a new message.
> >
> > With this said, is there a Node-Kafka client people are actually using
> in production that doesn't exhibit the profiles we have seen? We have
> back-tracked using Node-Kafka (v2.3) to only produce messages and rely on
> Redis PubSub channels for asynchronous acking of these messages. We would
> be willing to roll-up our sleeves with the community to develop a much more
> stable Node-Kafka client.
> >
> > Kind regards,
> >
> > Chris Alexander
> > Chief Technical Architect and Engineer
> > Gravy, Inc.
>
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB