Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> RE: partition and consumer stream number


+
김정민 2013-03-14, 02:00
+
永辉 赵 2012-12-19, 15:41
Copy link to this message
-
Re: partition and consumer stream number
Did you get my previous answer? I am including it below.

Currently, partition is the smallest unit that we distribute data among
consumers (in the same consumer group). So, if the # of consumers is larger
than the total number of partitions in a Kafka cluster (across all
brokers), some consumers will never get any data.

Thanks,

Jun

On Wed, Dec 19, 2012 at 3:07 AM, 永辉 赵 <[EMAIL PROTECTED]> wrote:

>
> Hi all,
>
> In consumer we use a topic count map to set how many stream we want to
> split
> the topic.
> What's the relation between kafka partition number and consumer stream
> number?
> Seems  stream number should be less or equal to partition number?
> Because if we set stream number bigger than partition number then some
> stream is empty or some partition is split.
>
> Thanks,
> Yonghui
>
>
>
>

 
+
Yonghui Zhao 2012-12-19, 16:42
+
Jun Rao 2012-12-19, 16:48
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB