Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Re: ConsumerRebalanceFailedException


Copy link to this message
-
Re: ConsumerRebalanceFailedException
Jun Rao 2013-12-30, 16:13
If the consumer is not shut down properly, it will take
zookeeper.session.timeout.ms before the consumer is deregistered from ZK.
If you restart the consumer before that, rebalances may fail.

Make sure that you call connector.shutdown() when you shut down the consumer

Thanks,

Jun
On Mon, Dec 30, 2013 at 1:58 AM, Hanish Bansal <
[EMAIL PROTECTED]> wrote:

> Hi All,
>
> I am getting consumer rebalance failed exception if i restart my consumer
> within 1-3 seconds.
>
> Exception trace is:
>
> Caused by: kafka.common.ConsumerRebalanceFailedException:
> indexConsumerGroup1_IMPETUS-I0027C-1388416992091-ac0d82d7 can't rebalance
> after 4 retries
>     at
>
> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(Unknown
> Source)
>     at
>
> kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(Unknown
> Source)
>     at kafka.consumer.ZookeeperConsumerConnector.consume(Unknown Source)
>     at
>
> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(Unknown
> Source)
>     at
>
> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(Unknown
> Source)
>
>
> Is this exception depends on any of below properties:
> zookeeper.session.timeout.ms  6000
>  zookeeper.connection.timeout.ms 6000
>
> If i kill the consumer and start that again after 5-6 sec then it started
> working properly without throwing any exception.
>
> If i start consumer immediately after killing that then
> ConsumerRebalanceFailedException occurs.
>
> Please help !!
>
> --
> *Thanks & Regards*
> *Hanish Bansal*
>