Sorry about the delayed reply to this, but it is an important point -
data can be lost even in this case. -1 means ack after all replicas in
the current ISR have received the message. So for example:
- assigned replicas for some partition = 0,1
- ISR = 0, leader = 0; (so 1 is out of the ISR - say if broker 0 is
slow (but up))
- messages with acks=-1 will get committed (i.e., the producer will
receive the ack)
- shutdown broker 0. So this will result in an unclean leader election
since there are no other replicas in ISR; and if the broker 1 is up it
will become the new leader
- however it only has messages until the earlier watermark
- when broker 0 comes back up it will become a follower to 1. Its
initial fetch will be for its last checkpointed high watermark (which
is larger than the current leader's hw), but the leader will piggy
back the current hw in the fetch response to the follower which will
then truncate its log to the leader's current hw. At least this is my
recollection of how it should be - but I'm staring at the code and
think there may be an issue with it. Will update this thread if this
is incorrect.



NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB