Search Hadoop and all its sub project:
Sort:

Switch to Threaded View
Subject: Re: at-least-once guarantee?
Interesting. .. wouldn't the producer sequence grow without bounds, in the
first case, even with the simpler non-ha of key assumption, to provide a
strict exactly once semantics?

In other words, wouldn't you need to store the entire set of keys that the
broker has ever seen to ensure that a potential replayed message doesn't
make it into the commit; given multiple producers?

In mps (github.com/milindparikh/mps), I use a rotating double bloom filter
to provide a "nearly exactly once" semantics to prevent an without-bound
growth of such a sequence.

Regards
Milind
On Aug 7, 2013 4:26 PM, "Jay Kreps" <[EMAIL PROTECTED]> wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB