If a consumer rebalances for any reason (e.g., if a consumer
in the group has a soft failure such as a long GC) then the fetchers
are stopped as part of the rebalance process. The sequence is as
- Stop fetchers
- Commit offsets
- Release partition ownership
- Rebalance (i.e., figure out what partitions this consumer should now
consume with the updated set of consumers)
- Acquire partition ownership
- Add fetchers to those partitions and resume consumption
i.e., rebalances should complete successfully and fetching should
resume. If you have any rebalance failures (search for "can't
rebalance after") then the consumer will effectively stop.
From later in this thread it seems your consumer somehow got into a
weird state in zookeeper, so your only recourse at this point may be
to stop all your consumers and restart.