-RE: are commitOffsets botched to zookeeper?
Neha Narkhede 2013-05-17, 13:18
Sorry I wasn't clear. Zookeeper 3.4.x has this feature. As soon as 08 is
stable and released it will be worth looking into when we can use zookeeper
On May 16, 2013 10:32 PM, "Rob Withers" <[EMAIL PROTECTED]> wrote:
> Can a request be made to zookeeper for this feature?
> > -----Original Message-----
> > From: Neha Narkhede [mailto:[EMAIL PROTECTED]]
> > Sent: Thursday, May 16, 2013 9:53 PM
> > To: [EMAIL PROTECTED]
> > Subject: Re: are commitOffsets botched to zookeeper?
> > Currently Kafka depends on zookeeper 3.3.4 that doesn't have a batch
> > api. So if you commit after every message at a high rate, it will be slow
> > inefficient. Besides it will cause zookeeper performance to degrade.
> > Thanks,
> > Neha
> > On May 16, 2013 6:54 PM, "Rob Withers" <[EMAIL PROTECTED]> wrote:
> > > We are calling commitOffsets after every message consumption. It
> > > looks to be ~60% slower, with 29 partitions. If a single KafkaStream
> > > thread is from a connector, and there are 29 partitions, then
> > > commitOffsets sends 29 offset updates, correct? Are these offset
> > > updates batched in one send to zookeeper?
> > >
> > > thanks,
> > > rob