Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Duplicate messages after new consumer introduction


Copy link to this message
-
Duplicate messages after new consumer introduction
navneet sharma 2012-05-16, 09:41
Hi,

I tried a scenario wherein:
1) i had 1 producer and 3 consumers subscribed for a topic - "cartTopic",
all in same group.
2) Now, when everything is executing, i introduced another consumer for the
same topic and in the same group. So, overall there are 4 consumers.
3) Ofcourse, it triggered re-balancing.

But then final result is that few messages are duplicated.
In my example run, producer sent 800,000 records, but consumer received
801,448 records.
I am using log4j to generate the output file.

Is there any reasons for duplicacy?

Thanks,
Navneet Sharma