Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Reg Kafka Replication


Copy link to this message
-
Re: Reg Kafka Replication
Hi Balasubramanian
Do you mean, why does Broker 0 stay in the replication set for partitions when it goes down?

This seems like a design decision as it means if 0 has a transient failure, due to a network blip or a restart, it will rejoin it's partitions when it comes back up. This would result in the least network activity as it would have most of the log already on disk and would just need to catch up.

One could also imagine a scenario where a broker went down because of load. This broker's partition assignments would get reassigned to other highly loaded brokers which could cause a cascading failure (I think).

You can reassign partitions with https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-6.ReassignPartitionsTool. It is also a very good idea to monitor the JMX bean for under replicated partitions as this would indicate the partitions at risk.

Daniel
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB