Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Offset committing on rebalance


Copy link to this message
-
Re: Offset committing on rebalance
Sorry, ignore that first exception, I believe that was caused by an actual manual shutdown. The NoNode exception though, has been popping up a lot, and I am not sure if it's relevant, but it seems to show up a bunch when the consumers decide it's time to rebalance continuously.  

--  
Ian Friedman
On Tuesday, August 20, 2013 at 2:17 AM, Ian Friedman wrote:

> That's not it either. I just had all my consumers shut down on me with this:  
>  
> INFO  21:51:13,948 () ZkUtils$ - conflict in /consumers/flurry1/owners/dataLogPaths/1-183 data: flurry1_hs1030-1376964634130-dcc9192a-0 stored data: flurry1_hs1061-1376964609207-4b7f348b-0
> INFO  21:51:13,948 () ZookeeperConsumerConnector - flurry1_hs1030-1376964634130-dcc9192a waiting for the partition ownership to be deleted: 1-183
> INFO  21:51:13,950 () ZookeeperConsumerConnector - flurry1_hs1030-1376964634130-dcc9192a flurry1_hs1030-1376964634130-dcc9192a-0 successfully owned partition 1-180 for topic dataLogPaths
>  
>  
>  
> and I've also been seeing:
>  
> INFO  21:51:15,971 () ZookeeperConsumerConnector - flurry1_hs1030-1376964634130-dcc9192a begin rebalancing consumer flurry1_hs1030-1376964634130-dcc9192a try #3
> INFO  21:51:16,038 () ZookeeperConsumerConnector - flurry1_hs1030-1376964634130-dcc9192a exception during rebalance  
> org.I0Itec.zkclient.exception.ZkNoNodeException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /consumers/flurry1/ids/flurry1_hs676-1376964612747-6f532caa
>         at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:47)
>         at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685)
>         at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:766)
>         at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:761)
>         at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:750)
>         at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:744)
>         at kafka.utils.ZkUtils$.readData(ZkUtils.scala:162)
>         at kafka.consumer.TopicCount$.constructTopicCount(TopicCount.scala:66)
>         at kafka.utils.ZkUtils$$anonfun$getConsumersPerTopic$1.apply(ZkUtils.scala:259)
>         at kafka.utils.ZkUtils$$anonfun$getConsumersPerTopic$1.apply(ZkUtils.scala:258)
>         at scala.collection.Iterator$class.foreach(Iterator.scala:631)
>         at scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:474)
>         at scala.collection.IterableLike$class.foreach(IterableLike.scala:79)
>         at scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:521)
>         at kafka.utils.ZkUtils$.getConsumersPerTopic(ZkUtils.scala:258)
>         at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.kafka$consumer$ZookeeperConsumerConnector$ZKRebalancerListener$$rebalance(ZookeeperConsumerConnector.scala:478)
>         at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala:449)
>         at scala.collection.immutable.Range$ByOne$class.foreach$mVc$sp(Range.scala:285)
>         at scala.collection.immutable.Range$$anon$2.foreach$mVc$sp(Range.scala:265)
>         at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:444)
>         at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$1.run(ZookeeperConsumerConnector.scala:401)
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /consumers/flurry1/ids/flurry1_hs676-1376964612747-6f532caa
>         at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
>         at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>         at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1151)
>         at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1180)
>         at org.I0Itec.zkclient.ZkConnection.readData(ZkConnection.java:103)
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB