Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Shutdown/Ctrl-C and ConsumerRebalanceFailedException


Copy link to this message
-
Re: Shutdown/Ctrl-C and ConsumerRebalanceFailedException
Chris Burroughs 2012-03-26, 02:01
If the consumer is cleanly shutdown (Control-C is kill, not kill -9,
right?), shoudn't we remove the ephemeral ZNodes in our shutdown
handler?  I thought this was already the case, at least for the broker.

On 03/20/2012 12:21 PM, Jun Rao wrote:
> Peter,
>
> When you kill your consumer, it takes sometime (by default 6 secs) for ZK
> server to release the ephemeral nodes hold by the consumer. If you restart
> your consumer quickly, the consumer may not be able to acquire the
> necessary zk nodes and therefore can fail to rebalance. One way is to
> implement a shutdown hook in your consumer application and make sure
> consumer connector is shut down when the consumer is killed.
>
> Thanks,
>
> Jun
>
> On Tue, Mar 20, 2012 at 2:42 AM, Peter Thygesen <[EMAIL PROTECTED]>wrote:
>
>> When I shutdown my consumer with crtl-c and tries to restart it quickly
>> afterwards, I usually get ConsumerRebalanceFailedException (see below). The
>> application then seems to hang.. or at least I'm sure if it is running any
>> more.. If this exception is thrown, will the consumer the intelligently
>> wait for the rebalancing to complete? and then resume consumption?
>>
>> I found a page
>> https://cwiki.apache.org/KAFKA/consumer-co-ordinator.htmlthat
>> describes something about Consumer Co-ordinator.. according to this
>> the consumer
>> group remains in this state until the next rebalancing attempt is
>> triggered. But when is it triggered?
>>
>> Could a shutdown hock with a consumer.commitOffsets help?
>> Does the consumer.shutdown implicit commitOffsets?
>>
>>
>> Exception in thread "main" kafka.common.ConsumerRebalanceFailedException:
>> contentItem-consumer-group-1_cphhdfs01node09-1332175323213-e6a3010f can't
>> rebalance after 4 retries
>>        at
>>
>> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:467)
>>        at
>>
>> kafka.consumer.ZookeeperConsumerConnector.consume(ZookeeperConsumerConnector.scala:204)
>>        at
>>
>> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:75)
>>        at
>>
>> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:89)
>>        at
>>
>> com.infopaq.research.repository.uima.ContentItemClient.consume(ContentItemClient.java:75)
>>        at
>>
>> com.infopaq.research.repository.uima.ContentItemClient.main(ContentItemClient.java:111)
>>
>>
>> Brgds,
>> Peter Thygesen
>>
>> BTW: Great work, very interesting project.
>>
>