I've recently been having problems with consumer groups rebalancing. I'm using several high level consumers which all belong to the same group. Occasionally one or two of them will get stuck in a rebalance loop. They attempt to rebalance, but the partitions they try to claim are owned. Anyone run into this? Ideas?
I see errors in my zookeeper logs like:
2013-12-17 17:12:31,171 [myid:001] - INFO [ProcessThread(sid:1 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x342e4febc180852 type:create cxid:0x1a9a zxid:0x501390d4b txntype:-1 reqpath:n/a Error Path:/kafka/consumers/trackingGroup/owners/Events2/25 Error:KeeperErrorCode = NodeExists for /kafka/consumers/trackingGroup/owners/Events2/25
The issue is that consumer 007 didn't see consumer 006 during rebalancing. So, it made a decision in conflict with consumer 006. Consumer 007 should have another ZK watcher fired to trigger another rebalance when if it will see consumer 006. Which version of ZK are you using?
Jun On Wed, Dec 18, 2013 at 9:38 AM, Drew Goya <[EMAIL PROTECTED]> wrote:
We recently upgraded to 3.4.5, so far without incident. But I'd be interested to know whether we confirm that there are known problems with this!
Jason On Mon, Dec 23, 2013 at 2:04 PM, Drew Goya <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation project and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext