-Re: Duplicate messages after new consumer introduction
navneet sharma 2012-05-17, 06:45
I downloaded the tar from the download link provided in quickstart page.
Almost more than a month back.
I trunk maintaining different code than the tar?
Can number of partitions cause this problem, beacuse i am using 2
partitions on each of the two brokers.?
On Wed, May 16, 2012 at 10:00 PM, Jay Kreps <[EMAIL PROTECTED]> wrote:
> Technically this is the guarantee we provide--at least once delivery.
> It is very expensive to completely eliminate this possibility in the
> general case as you need to co-ordinate any state changes the consumer
> makes with committing the offset that marks the position. But we have
> improved the common cases for normal rebalancing so if you are using
> trunk the only time this would happen is when there is a hard crash of
> a process.
> On Wed, May 16, 2012 at 2:41 AM, navneet sharma
> <[EMAIL PROTECTED]> wrote:
> > Hi,
> > I tried a scenario wherein:
> > 1) i had 1 producer and 3 consumers subscribed for a topic - "cartTopic",
> > all in same group.
> > 2) Now, when everything is executing, i introduced another consumer for
> > same topic and in the same group. So, overall there are 4 consumers.
> > 3) Ofcourse, it triggered re-balancing.
> > But then final result is that few messages are duplicated.
> > In my example run, producer sent 800,000 records, but consumer received
> > 801,448 records.
> > I am using log4j to generate the output file.
> > Is there any reasons for duplicacy?
> > Thanks,
> > Navneet Sharma