Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> consumer partition rebalancing

Copy link to this message
consumer partition rebalancing
I wasreviewingtheconsumer partition rebalancingalgorithm and had a fewrelatedquestions 

* Assuming 1) theconsumerdoesn'tcontrolthe partition allocation within a topic and 2) theconstraintthat a single consumer C(i) within a consumer group C(g) must be the only reader of that partition:
* Are there ways to scale partition consumption if C(i) cannot keep up?
* A way to designate particular consumers should consume particular partitions based on consumer capability (similar to container allocation in YARN)?
* A way to designate sticky consumer / partition allocation so that a new consumer C(j) or new broker B(k) or new topic T(l) does not cause C(i) to lose particular partitions during rebalancing?