Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Number of consumers limited to number of partitions


Copy link to this message
-
Number of consumers limited to number of partitions
Hi all,

We have a topic with three partitions - one on each broker (Kafka 0.7.1).
Using ConsumerConnector I noticed that I can consume messages from three
consumers at most. When I start fourth one, one of the older one stops.

Is this as designed? If I want more parallelism, do I need to increase
number of partitions?

Regards,
Vaibhav
GumGum
+
Neha Narkhede 2012-10-11, 15:27
+
Jay Kreps 2012-10-11, 15:41
+
Vaibhav Puranik 2012-10-11, 16:34
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB