-Re: redis versus zookeeper to track consumer offsets
Neha Narkhede 2013-12-18, 00:37
You can do this indirectly by monitoring the avg/max latency of operations
on zookeeper. There is no direct way of measuring the requests/sec to
zookeeper since they don't expose the relevant jmx metrics.
On Tue, Dec 17, 2013 at 11:13 AM, S Ahmed <[EMAIL PROTECTED]> wrote:
> Interesting, wasn't aware of that.
> Can you comment on how you go about monitoring your ZK cluster in terms of
> throughput and if it is reaching its limits? Or is it even possible to do
> On Tue, Dec 17, 2013 at 2:01 PM, Benjamin Black <[EMAIL PROTECTED]> wrote:
> > ZK was designed from the start as a clustered, consistent, highly
> > store for this sort of data and it works extremely well. Redis wasn't
> and I
> > don't know anyone using Redis in production, including me, who doesn't
> > stories of Redis losing data. I'm sticking with ZK.
> > On Tue, Dec 17, 2013 at 10:57 AM, S Ahmed <[EMAIL PROTECTED]> wrote:
> > > I am leaning towards using redis to track consumer offsets etc., but I
> > see
> > > how using zookeeper makes sense since it already part of the kafka
> > >
> > > One thing which bothers me is, how are you guys keeping track of the
> > > on zookeeper? How do you get an idea when your zookeeper cluster is
> > > underprovisioned?
> > >
> > > Redis is a richer store and could help in other areas where you want to
> > > store more than just status information or offsets, and setup and
> > > administration wise it seems a bit easier to manage.
> > >
> > > Thoughts?
> > >