Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Question about kafka consumer stream/partition


+
Bae, Jae Hyeon 2013-01-14, 22:06
Copy link to this message
-
Re: Question about kafka consumer stream/partition
> My question is, does the number of kafka consumers mean the number of
> kafka streams?
>

Yes. To know the total number of consumers/streams in a group, you need to
add up the number of streams on every consumer instance
> For example, I have one broker with one partition. What if I create
> the consumer with 4 streams? Will 3 streams be idle? Or Will 4 streams
> get the split data from the one partition?
>

3 streams will be idle. Partition is the smallest granularity of
consumption. This is because if we allow streams to share partitions, there
will be a lot of distributed locking involved affecting throughput
adversely.

Thanks
Neha

 
+
Baran Nohutçuoğlu 2014-06-26, 21:15
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB