Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Re: Reg Kafka Replication


Copy link to this message
-
Re: Reg Kafka Replication
Daniel Compton 2014-06-30, 06:14
Hi Balasubramanian
Do you mean, why does Broker 0 stay in the replication set for partitions when it goes down?

This seems like a design decision as it means if 0 has a transient failure, due to a network blip or a restart, it will rejoin it's partitions when it comes back up. This would result in the least network activity as it would have most of the log already on disk and would just need to catch up.

One could also imagine a scenario where a broker went down because of load. This broker's partition assignments would get reassigned to other highly loaded brokers which could cause a cascading failure (I think).

You can reassign partitions with https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-6.ReassignPartitionsTool. It is also a very good idea to monitor the JMX bean for under replicated partitions as this would indicate the partitions at risk.

Daniel