Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> redis versus zookeeper to track consumer offsets


Copy link to this message
-
Re: redis versus zookeeper to track consumer offsets
ZK was designed from the start as a clustered, consistent, highly available
store for this sort of data and it works extremely well. Redis wasn't and I
don't know anyone using Redis in production, including me, who doesn't have
stories of Redis losing data. I'm sticking with ZK.
On Tue, Dec 17, 2013 at 10:57 AM, S Ahmed <[EMAIL PROTECTED]> wrote:

> I am leaning towards using redis to track consumer offsets etc., but I see
> how using zookeeper makes sense since it already part of the kafka infra.
>
> One thing which bothers me is, how are you guys keeping track of the load
> on zookeeper?  How do you get an idea when your zookeeper cluster is
> underprovisioned?
>
> Redis is a richer store and could help in other areas where you want to
> store more than just status information or offsets, and setup and
> administration wise it seems a bit easier to manage.
>
> Thoughts?
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB