Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Kafka client dies after rebalancing attempt


Copy link to this message
-
Re: Kafka client dies after rebalancing attempt
Can you see if this applies in your case:
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyaretheremanyrebalancesinmyconsumerlog%3F

Also, what version of kafka 0.8 are you using? If not the beta, then
what's the git hash?

Joel
 
On Thu, Nov 07, 2013 at 02:51:41PM -0500, Ahmed H. wrote:
> Hello all,
>
> I am not sure if this is a Kafka issue, or an issue with the client that I
> am using.
>
> We have a fairly small setup, where everything sits on one server (Kafka
> 0.8, and Zookeeper). The message frequency is not too high (1-2 per second).
>
> The setup works fine for a certain period of time but at some point, it
> just dies, and exceptions are thrown. This is pretty much a daily
> occurrence, but there is no pattern. Based on the logs, it appears that the
> Kafka client tries to rebalance with Zookeeper and fails, it tries and
> tries multiple times but after a few tries it gives up. Here is the stack
> trace:
>
> 04:56:07,234 INFO  [kafka.consumer.SimpleConsumer]
> > (ConsumerFetcherThread-kafkaqueue.notifications_test-server.localnet-1383643783745-3757e7a5-0-0)
> > Reconnect due to socket error: :
> > java.nio.channels.ClosedByInterruptException
> >  at
> > java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
> > [rt.jar:1.7.0_25]
> > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:402)
> > [rt.jar:1.7.0_25]
> >  at
> > sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:220)
> > [rt.jar:1.7.0_25]
> > at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> > [rt.jar:1.7.0_25]
> >  at
> > java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
> > [rt.jar:1.7.0_25]
> > at kafka.utils.Utils$.read(Utils.scala:394)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> >  at
> > kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> > at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> >  at
> > kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> > at kafka.network.BlockingChannel.receive(BlockingChannel.scala:100)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> >  at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:71)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> > at
> > kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:69)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> >  at
> > kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:108)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> > at
> > kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:108)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> >  at
> > kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:108)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> > at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> >  at
> > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:107)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> > at
> > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:107)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> >  at
> > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:107)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> > at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> >  at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:106)
> > [kafka_2.9.2-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT]
> > at
> > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:96)