Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Consumer Group Rebalance Issues


Copy link to this message
-
Re: Consumer Group Rebalance Issues
We recently upgraded to 3.4.5, so far without incident.  But I'd be
interested to know whether we confirm that there are known problems with
this!

Jason
On Mon, Dec 23, 2013 at 2:04 PM, Drew Goya <[EMAIL PROTECTED]> wrote:

> Thanks, I migrated our ZK cluster over to 3.3 this weekend.  Hopefully that
> does it!
>
>
> On Fri, Dec 20, 2013 at 9:09 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Hmm, not sure how stable 3.4.4. We have been using 3.3.4 and haven't seen
> > issues with ZK as long as there aren't many ZK session expirations.
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, Dec 19, 2013 at 9:41 PM, Drew Goya <[EMAIL PROTECTED]> wrote:
> >
> > > Our cluster is currently running 3.4.4.
> > >
> > > I see Kafka is currently using the 3.3.4 client, is there a potential
> > > conflict there?
> > >
> > >
> > > On Wed, Dec 18, 2013 at 9:12 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > >
> > > > The issue is that consumer 007 didn't see consumer 006 during
> > > rebalancing.
> > > > So, it made a decision in conflict with consumer 006. Consumer 007
> > should
> > > > have another ZK watcher fired to trigger another rebalance when if it
> > > will
> > > > see consumer 006. Which version of ZK are you using?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > >
> > > > On Wed, Dec 18, 2013 at 9:38 AM, Drew Goya <[EMAIL PROTECTED]>
> wrote:
> > > >
> > > > > Thanks for the help with this Jun, really appreciate it!  So I
> found
> > > this
> > > > > in the logs for consumer 007 about an hour previous.  Besides that
> no
> > > > real
> > > > > activity.
> > > > >
> > > > > It looks like 007 rebalanced and successfully claimed partition
> > 24-27.
> > > > >  Shortly after that its zookeeper client timed out and reconnected.
> >  It
> > > > > didn't rebalance again after this.
> > > > >
> > > > > 2013-12-17 15:51:06 ZookeeperConsumerConnector [INFO]
> > > > > [trackingGroup_prod-storm-sup-trk007-1387249529436-fb79e4c8], begin
> > > > > rebalancing consumer
> > > > > trackingGroup_prod-storm-sup-trk007-1387249529436-fb79e4c8 try #0
> > > > > 2013-12-17 15:51:06 ConsumerFetcherManager [INFO]
> > > > > [ConsumerFetcherManager-1387249529483] Stopping leader finder
> thread
> > > > > 2013-12-17 15:51:06 ConsumerFetcherManager$LeaderFinderThread
> [INFO]
> > > > >
> > > > >
> > > >
> > >
> >
> [trackingGroup_prod-storm-sup-trk007-1387249529436-fb79e4c8-leader-finder-thread],
> > > > > Shutting down
> > > > > 2013-12-17 15:51:06 ConsumerFetcherManager$LeaderFinderThread
> [INFO]
> > > > >
> > > > >
> > > >
> > >
> >
> [trackingGroup_prod-storm-sup-trk007-1387249529436-fb79e4c8-leader-finder-thread],
> > > > > Stopped
> > > > > 2013-12-17 15:51:06 ConsumerFetcherManager$LeaderFinderThread
> [INFO]
> > > > >
> > > > >
> > > >
> > >
> >
> [trackingGroup_prod-storm-sup-trk007-1387249529436-fb79e4c8-leader-finder-thread],
> > > > > Shutdown completed
> > > > > 2013-12-17 15:51:06 ConsumerFetcherManager [INFO]
> > > > > [ConsumerFetcherManager-1387249529483] Stopping all fetchers
> > > > > 2013-12-17 15:51:06 ConsumerFetcherThread [INFO]
> > > > >
> > > > >
> > > >
> > >
> >
> [ConsumerFetcherThread-trackingGroup_prod-storm-sup-trk007-1387249529436-fb79e4c8-0-13],
> > > > > Shutting down
> > > > > 2013-12-17 15:51:06 ConsumerFetcherThread [INFO]
> > > > >
> > > > >
> > > >
> > >
> >
> [ConsumerFetcherThread-trackingGroup_prod-storm-sup-trk007-1387249529436-fb79e4c8-0-13],
> > > > > Stopped
> > > > > 2013-12-17 15:51:06 ConsumerFetcherThread [INFO]
> > > > >
> > > > >
> > > >
> > >
> >
> [ConsumerFetcherThread-trackingGroup_prod-storm-sup-trk007-1387249529436-fb79e4c8-0-13],
> > > > > Shutdown completed
> > > > > 2013-12-17 15:51:06 ConsumerFetcherThread [INFO]
> > > > >
> > > > >
> > > >
> > >
> >
> [ConsumerFetcherThread-trackingGroup_prod-storm-sup-trk007-1387249529436-fb79e4c8-0-11],
> > > > > Shutting down
> > > > > 2013-12-17 15:51:06 SimpleConsumer [INFO] Reconnect due to socket
> > > error:
> > > > > null
> > > > > 2013-12-17 15:51:06 ConsumerFetcherThread [INFO]