Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> [jira] [Updated] (KAFKA-1072) Allow mulitple topics selected with a TopicFilter to be balanced among consumers


Copy link to this message
-
[jira] [Updated] (KAFKA-1072) Allow mulitple topics selected with a TopicFilter to be balanced among consumers

     [ https://issues.apache.org/jira/browse/KAFKA-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Neha Narkhede updated KAFKA-1072:
---------------------------------

    Fix Version/s: 0.9

> Allow mulitple topics selected with a TopicFilter to be balanced among consumers
> --------------------------------------------------------------------------------
>
>                 Key: KAFKA-1072
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1072
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8
>            Reporter: Jason Rosenberg
>             Fix For: 0.9
>
>
> Currently, there is no parallelism used when consuming a set of topics selected by a white list topic filter, if those topics all have a partition count of 1.  Currently, all topics that match the filter get assigned to the same thread on the same consumer, even though there may be plenty of different topics (and therefore many partitions to be consumed from).
> There are often good reasons to use a partition count of only 1 (e.g. to preserve message ordering).  For arbitrary scalability, over a large number of topics, this would be a great benefit to be able to consume topics balanced over a set of available consumers.

--
This message was sent by Atlassian JIRA
(v6.1#6144)

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB