Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Question about kafka consumer stream/partition


Copy link to this message
-
Re: Question about kafka consumer stream/partition
Neha Narkhede 2013-01-15, 00:16
> My question is, does the number of kafka consumers mean the number of
> kafka streams?
>

Yes. To know the total number of consumers/streams in a group, you need to
add up the number of streams on every consumer instance
> For example, I have one broker with one partition. What if I create
> the consumer with 4 streams? Will 3 streams be idle? Or Will 4 streams
> get the split data from the one partition?
>

3 streams will be idle. Partition is the smallest granularity of
consumption. This is because if we allow streams to share partitions, there
will be a lot of distributed locking involved affecting throughput
adversely.

Thanks
Neha