Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Re: ordering


Copy link to this message
-
Re: ordering
Jun Rao 2013-08-22, 04:01
Actually, I am not sure if I understand the trouble that you mentioned.
Could you elaborate that a bit more?

Thanks,

Jun
On Wed, Aug 21, 2013 at 12:30 PM, Yu, Libo <[EMAIL PROTECTED]> wrote:

> Hi,
>
> This is from kafka faq:
>
>
>   *   Each partition is not consumed by more than one consumer
> thread/process in each consumer group. This allows to have each process
> consume in a single threaded fashion to guarantee ordering to the consumer
> within the partition (if we split up a partition of ordered messages and
> handed them out to multiple consumers even though the messages were stored
> in order they would be processed out of order at times).
>
> Is this doable?
>
> Say a topic has 3 partitions. And there are 3 consumer processes in a
> consumer group each of which has a single thread.
> When we start the 3 processes, when the first process is up, it will
> consume all 3 partitions. When the second process is up,
> 1 process consumes 2 partitions and 1 consumes 1 partition. Only when the
> third process is up, will each process consume
> One partition. This will cause trouble. There seems to be no way to bound
> a stream to one partition.
>
>
> Regards,
>
> Libo
>
>

 
+
Philip OToole 2013-08-21, 20:12
+
Jun Rao 2013-08-22, 14:42