Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - redis versus zookeeper to track consumer offsets


Copy link to this message
-
Re: redis versus zookeeper to track consumer offsets
Benjamin Black 2013-12-17, 19:02
ZK was designed from the start as a clustered, consistent, highly available
store for this sort of data and it works extremely well. Redis wasn't and I
don't know anyone using Redis in production, including me, who doesn't have
stories of Redis losing data. I'm sticking with ZK.
On Tue, Dec 17, 2013 at 10:57 AM, S Ahmed <[EMAIL PROTECTED]> wrote:

> I am leaning towards using redis to track consumer offsets etc., but I see
> how using zookeeper makes sense since it already part of the kafka infra.
>
> One thing which bothers me is, how are you guys keeping track of the load
> on zookeeper?  How do you get an idea when your zookeeper cluster is
> underprovisioned?
>
> Redis is a richer store and could help in other areas where you want to
> store more than just status information or offsets, and setup and
> administration wise it seems a bit easier to manage.
>
> Thoughts?
>