Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Consumer resetting if zookeeper folder deleted


Copy link to this message
-
Re: Consumer resetting if zookeeper folder deleted
Hi Neha,

if i delete the zookeper folder, then i can replicate this problem.
Actually, that folder was getting created in "/tmp" folder and it was
getting deleted each day. Now i changed the location and i am seeing that
the exception is gone.

Basically, i was just checking if i restart the consumer, say after 2-3
days, where it will start consuming from. My assumption was that this
information should be part of broker/server but looks like its part of
zookeeper and thats why i observed these exceptions.

Attaching logs for your reference.

Thanks,
Navneet Sharma

On Tue, May 8, 2012 at 12:06 AM, Neha Narkhede <[EMAIL PROTECTED]>wrote:

> >> Is it an expected behaviour?
>
> Not really. That exception means that the Zookeeper server your client is
> connected to, is trying to send a response to the client's "ping", but
> failed, since probably the client disconnected from the server already.
> I'm not sure I understand what led to this. Could you attach the entire
> kafka + zookeeper log here ?
>
> Thanks,
> Neha
>
>
> On Mon, May 7, 2012 at 8:18 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > ZK stores critical data such as topics/partitions on each broker, in
> > addition to consumer offsets. So ZK data needs to be protected.
> >
> > Thanks,
> >
> > Jun
> >
> > On Mon, May 7, 2012 at 4:21 AM, navneet sharma
> > <[EMAIL PROTECTED]>wrote:
> >
> > > Hi,
> > >
> > > I observed that if zookeeper folder is deleted, consumer is trying to
> > read
> > > the messages it read in previous runs. And there are lots of exceptions
> > > seen on zookeeper console as well::
> > >
> > > java.nio.channels.CancelledKeyException
> > >    at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
> > >    at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:59)
> > >    at
> > >
> > >
> >
> org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:418)
> > >    at
> > >
> > >
> >
> org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1509)
> > >    at
> > >
> > >
> >
> org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:171)
> > >    at
> > >
> > >
> >
> org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:135)
> > > [2012-05-07 15:57:35,569] ERROR Unexpected Exception:
> > > (org.apache.zookeeper.server.NIOServerCnxn)
> > >
> > >
> > > Is it an expected behaviour? Is zookeeper directory need to be
> protected
> > to
> > > avoid consumer falling off on offsets?
> > >
> > > Thanks,
> > > Navneet Sharma
> > >
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB