Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: Reg Partition


Copy link to this message
-
Re: Reg Partition
You can certainly have several consumers consuming from the same partition: just give each a different consumer group ID, and then all the messages from the partition will be delivered to all of the consumers.

If you want each message to only be processed by one of the consumers, you can drop those that you don't want: for example, consumer 1 ignores all messages with an even-numbered offset, and consumer 2 ignores all messages with an odd-numbered offset.

However, I don't understand why you want to have multiple consumers on the same partition in the first place. Can't you simply configure your topic to have enough partitions that you can achieve the required parallelism? That's what partitions are for.

Martin

On 6 Mar 2014, at 01:19, Balasubramanian Jayaraman (Contingent) <[EMAIL PROTECTED]> wrote: