Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Duplicate messages after new consumer introduction

Copy link to this message
Duplicate messages after new consumer introduction

I tried a scenario wherein:
1) i had 1 producer and 3 consumers subscribed for a topic - "cartTopic",
all in same group.
2) Now, when everything is executing, i introduced another consumer for the
same topic and in the same group. So, overall there are 4 consumers.
3) Ofcourse, it triggered re-balancing.

But then final result is that few messages are duplicated.
In my example run, producer sent 800,000 records, but consumer received
801,448 records.
I am using log4j to generate the output file.

Is there any reasons for duplicacy?

Navneet Sharma