"Bae, Jae Hyeon" 2013-01-14, 22:06
-Re: Question about kafka consumer stream/partition
Neha Narkhede 2013-01-15, 00:16
> My question is, does the number of kafka consumers mean the number of
> kafka streams?
Yes. To know the total number of consumers/streams in a group, you need to
add up the number of streams on every consumer instance
> For example, I have one broker with one partition. What if I create
> the consumer with 4 streams? Will 3 streams be idle? Or Will 4 streams
> get the split data from the one partition?
3 streams will be idle. Partition is the smallest granularity of
consumption. This is because if we allow streams to share partitions, there
will be a lot of distributed locking involved affecting throughput