Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Shutdown/Ctrl-C and ConsumerRebalanceFailedException


Copy link to this message
-
Re: Shutdown/Ctrl-C and ConsumerRebalanceFailedException
If the consumer is cleanly shutdown (Control-C is kill, not kill -9,
right?), shoudn't we remove the ephemeral ZNodes in our shutdown
handler?  I thought this was already the case, at least for the broker.

On 03/20/2012 12:21 PM, Jun Rao wrote:
> Peter,
>
> When you kill your consumer, it takes sometime (by default 6 secs) for ZK
> server to release the ephemeral nodes hold by the consumer. If you restart
> your consumer quickly, the consumer may not be able to acquire the
> necessary zk nodes and therefore can fail to rebalance. One way is to
> implement a shutdown hook in your consumer application and make sure
> consumer connector is shut down when the consumer is killed.
>
> Thanks,
>
> Jun
>
> On Tue, Mar 20, 2012 at 2:42 AM, Peter Thygesen <[EMAIL PROTECTED]>wrote:
>
>> When I shutdown my consumer with crtl-c and tries to restart it quickly
>> afterwards, I usually get ConsumerRebalanceFailedException (see below). The
>> application then seems to hang.. or at least I'm sure if it is running any
>> more.. If this exception is thrown, will the consumer the intelligently
>> wait for the rebalancing to complete? and then resume consumption?
>>
>> I found a page
>> https://cwiki.apache.org/KAFKA/consumer-co-ordinator.htmlthat
>> describes something about Consumer Co-ordinator.. according to this
>> the consumer
>> group remains in this state until the next rebalancing attempt is
>> triggered. But when is it triggered?
>>
>> Could a shutdown hock with a consumer.commitOffsets help?
>> Does the consumer.shutdown implicit commitOffsets?
>>
>>
>> Exception in thread "main" kafka.common.ConsumerRebalanceFailedException:
>> contentItem-consumer-group-1_cphhdfs01node09-1332175323213-e6a3010f can't
>> rebalance after 4 retries
>>        at
>>
>> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:467)
>>        at
>>
>> kafka.consumer.ZookeeperConsumerConnector.consume(ZookeeperConsumerConnector.scala:204)
>>        at
>>
>> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:75)
>>        at
>>
>> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:89)
>>        at
>>
>> com.infopaq.research.repository.uima.ContentItemClient.consume(ContentItemClient.java:75)
>>        at
>>
>> com.infopaq.research.repository.uima.ContentItemClient.main(ContentItemClient.java:111)
>>
>>
>> Brgds,
>> Peter Thygesen
>>
>> BTW: Great work, very interesting project.
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB