Kafka, mail # user - Re: at-least-once guarantee? - 2013-08-08, 00:50
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: at-least-once guarantee?
Interesting. .. wouldn't the producer sequence grow without bounds, in the
first case, even with the simpler non-ha of key assumption, to provide a
strict exactly once semantics?

In other words, wouldn't you need to store the entire set of keys that the
broker has ever seen to ensure that a potential replayed message doesn't
make it into the commit; given multiple producers?

In mps (github.com/milindparikh/mps), I use a rotating double bloom filter
to provide a "nearly exactly once" semantics to prevent an without-bound
growth of such a sequence.

Regards
Milind
On Aug 7, 2013 4:26 PM, "Jay Kreps" <[EMAIL PROTECTED]> wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB