Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Question about kafka consumer stream/partition

Bae, Jae Hyeon 2013-01-14, 22:06
Copy link to this message
Re: Question about kafka consumer stream/partition
> My question is, does the number of kafka consumers mean the number of
> kafka streams?

Yes. To know the total number of consumers/streams in a group, you need to
add up the number of streams on every consumer instance
> For example, I have one broker with one partition. What if I create
> the consumer with 4 streams? Will 3 streams be idle? Or Will 4 streams
> get the split data from the one partition?

3 streams will be idle. Partition is the smallest granularity of
consumption. This is because if we allow streams to share partitions, there
will be a lot of distributed locking involved affecting throughput


Baran Nohutçuoğlu 2014-06-26, 21:15