Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> RE: partition and consumer stream number


+
김정민 2013-03-14, 02:00
+
永辉 赵 2012-12-19, 15:41
Copy link to this message
-
Re: partition and consumer stream number
Did you get my previous answer? I am including it below.

Currently, partition is the smallest unit that we distribute data among
consumers (in the same consumer group). So, if the # of consumers is larger
than the total number of partitions in a Kafka cluster (across all
brokers), some consumers will never get any data.

Thanks,

Jun

On Wed, Dec 19, 2012 at 3:07 AM, 永辉 赵 <[EMAIL PROTECTED]> wrote:

>
> Hi all,
>
> In consumer we use a topic count map to set how many stream we want to
> split
> the topic.
> What's the relation between kafka partition number and consumer stream
> number?
> Seems  stream number should be less or equal to partition number?
> Because if we set stream number bigger than partition number then some
> stream is empty or some partition is split.
>
> Thanks,
> Yonghui
>
>
>
>

 
+
Yonghui Zhao 2012-12-19, 16:42
+
Jun Rao 2012-12-19, 16:48