Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # dev >> [jira] [Assigned] (KAFKA-1050) Support for "no data loss" mode


Copy link to this message
-
[jira] [Assigned] (KAFKA-1050) Support for "no data loss" mode

     [ https://issues.apache.org/jira/browse/KAFKA-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Neha Narkhede reassigned KAFKA-1050:
------------------------------------

    Assignee: Neha Narkhede

> Support for "no data loss" mode
> -------------------------------
>
>                 Key: KAFKA-1050
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1050
>             Project: Kafka
>          Issue Type: Task
>            Reporter: Justin SB
>            Assignee: Neha Narkhede
>
> I'd love to use Apache Kafka, but for my application data loss is not acceptable.  Even at the expense of availability (i.e. I need C not A in CAP).
> I think there are two things that I need to change to get a quorum model:
> 1) Make sure I set request.required.acks to 2 (for a 3 node cluster) or 3 (for a 5 node cluster) on every request, so that I can only write if a quorum is active.
> 2) Prevent the behaviour where a non-ISR can become the leader if all ISRs die.  I think this is as easy as tweaking core/src/main/scala/kafka/controller/PartitionLeaderSelector.scala, essentially to throw an exception around line 64 in the "data loss" case.
> I haven't yet implemented / tested this.  I'd love to get some input from the Kafka-experts on whether my plan is:
>  (a) correct - will this work?
>  (b) complete - have I missed any cases?
>  (c) recommended - is this a terrible idea :-)
> Thanks for any pointers!

--
This message was sent by Atlassian JIRA
(v6.1#6144)

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB