Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> redis versus zookeeper to track consumer offsets

Copy link to this message
Re: redis versus zookeeper to track consumer offsets
Interesting, wasn't aware of that.

Can you comment on how you go about monitoring your ZK cluster in terms of
throughput and if it is reaching its limits? Or is it even possible to do
On Tue, Dec 17, 2013 at 2:01 PM, Benjamin Black <[EMAIL PROTECTED]> wrote:

> ZK was designed from the start as a clustered, consistent, highly available
> store for this sort of data and it works extremely well. Redis wasn't and I
> don't know anyone using Redis in production, including me, who doesn't have
> stories of Redis losing data. I'm sticking with ZK.
> On Tue, Dec 17, 2013 at 10:57 AM, S Ahmed <[EMAIL PROTECTED]> wrote:
> > I am leaning towards using redis to track consumer offsets etc., but I
> see
> > how using zookeeper makes sense since it already part of the kafka infra.
> >
> > One thing which bothers me is, how are you guys keeping track of the load
> > on zookeeper?  How do you get an idea when your zookeeper cluster is
> > underprovisioned?
> >
> > Redis is a richer store and could help in other areas where you want to
> > store more than just status information or offsets, and setup and
> > administration wise it seems a bit easier to manage.
> >
> > Thoughts?
> >