-How kafka assign partition to stream
Yonghui Zhao 2014-03-10, 17:23
In my environment, I have 2 brokers and only 1 topic, each brokers has 10
so there are 20 partitions in total.
I have 4 consumers in one consumer group, each consumer use
createMessageStreams to create 10 streams, 40 streams in total.
Since partition can not be split, so there are 20 streams are empty
We found 2 machines have 10 valid streams, but the other 2 have no valid
Is it expected? Maybe each consumer has 5 valid streams is better for load
And I want to know if one of the machines which have 10 valid streams is
dead, will the the partitions be balanced to other machine dynamically?
i.e. some empty stream in other machines will get data now.