Hi

I know if the number of kafka consumers is greater than the number of
partitions in the kafka broker cluster, several kafka consumers will
be idle.

My question is, does the number of kafka consumers mean the number of
kafka streams?

For example, I have one broker with one partition. What if I create
the consumer with 4 streams? Will 3 streams be idle? Or Will 4 streams
get the split data from the one partition?

Thank you
Best, Jae

 

> My question is, does the number of kafka consumers mean the number of
> kafka streams?
>

Yes. To know the total number of consumers/streams in a group, you need to
add up the number of streams on every consumer instance
> For example, I have one broker with one partition. What if I create
> the consumer with 4 streams? Will 3 streams be idle? Or Will 4 streams
> get the split data from the one partition?
>

3 streams will be idle. Partition is the smallest granularity of
consumption. This is because if we allow streams to share partitions, there
will be a lot of distributed locking involved affecting throughput
adversely.

Thanks
Neha