Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Kafka client dies after rebalancing attempt


Copy link to this message
-
Re: Kafka client dies after rebalancing attempt
Can you see if this applies in your case:
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyaretheremanyrebalancesinmyconsumerlog%3F

Also, what version of kafka 0.8 are you using? If not the beta, then
what's the git hash?

Joel
 
On Thu, Nov 07, 2013 at 02:51:41PM -0500, Ahmed H. wrote:
> Hello all,
>
> I am not sure if this is a Kafka issue, or an issue with the client that I
> am using.
>
> We have a fairly small setup, where everything sits on one server (Kafka
> 0.8, and Zookeeper). The message frequency is not too high (1-2 per second).
>
> The setup works fine for a certain period of time but at some point, it
> just dies, and exceptions are thrown. This is pretty much a daily
> occurrence, but there is no pattern. Based on the logs, it appears that the
> Kafka client tries to rebalance with Zookeeper and fails, it tries and
> tries multiple times but after a few tries it gives up. Here is the stack
> trace:
>
> 04:56:07,234 INFO  [kafka.consumer.SimpleConsumer]
> > (ConsumerFetcherThread-kafkaqueue.notifications_test-server.localnet-1383643783745-3757e7a5-0-0)
> > Reconnect due to socket error: :
> > java.nio.channels.ClosedByInterruptException
> >  at
> > java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
> > [rt.jar:1.7.0_25]
> > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:402)
> > [rt.jar:1.7.0_25]
> >  at
> > sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:220)
> > [rt.jar:1.7.0_25]
> > at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> > [rt.jar:1.7.0_25]
> >  at
> > java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
> > [rt.jar:1.7.0_25]
> > at kafka.utils.Utils$.read(Utils.scala:394)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> >  at
> > kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> > at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> >  at
> > kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> > at kafka.network.BlockingChannel.receive(BlockingChannel.scala:100)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> >  at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:71)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> > at
> > kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:69)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> >  at
> > kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:108)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> > at
> > kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:108)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> >  at
> > kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:108)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> > at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> >  at
> > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:107)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> > at
> > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:107)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> >  at
> > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:107)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> > at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> >  at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:106)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> > at
> > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:96)
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB