Kafka, mail # user - Re: Data loss in case of request.required.acks set to -1 - 2013-12-23, 06:39
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: Data loss in case of request.required.acks set to -1
Hi Guazhang,

When both nodes are alive then topic isr status is:

topic: test-trunk111    partition: 0    leader: 0    replicas: 1,0    isr: 0
topic: test-trunk111    partition: 1    leader: 0    replicas: 0,1    isr: 0

Now as the leader node is broker-0 so when i am producing the data then
meanwhile kill the leader node.
After leader goes down, topic isr status is:

topic: test-trunk111    partition: 0    leader: 1    replicas: 1,0    isr: 1
topic: test-trunk111    partition: 1    leader: 1    replicas: 0,1    isr: 1

Now after all data produced when i consumed the data, there is some data
loss.

*Also in controller logs there is entry like:*

[2013-12-23 10:25:07,648] DEBUG [OfflinePartitionLeaderSelector]: No broker
in ISR is alive for [test-trunk111,1]. Pick the leader from the alive
assigned replicas: 1 (kafka.controller.OfflinePartitionLeaderSelector)
[2013-12-23 10:25:07,648] WARN [OfflinePartitionLeaderSelector]: No broker
in ISR is alive for [test-trunk111,1]. Elect leader 1 from live brokers 1.
There's potential data loss.
(kafka.controller.OfflinePartitionLeaderSelector)
[2013-12-23 10:25:07,649] INFO [OfflinePartitionLeaderSelector]: Selected
new leader and ISR {"leader":1,"leader_epoch":1,"isr":[1]} for offline
partition [test-trunk111,1]
(kafka.controller.OfflinePartitionLeaderSelector)

Is there any solution for this behaviour ?
On Fri, Dec 20, 2013 at 7:27 PM, Guozhang Wang <[EMAIL PROTECTED]> wrote:
*Thanks & Regards*
*Hanish Bansal*

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB