Search Hadoop and all its sub project:

Switch to Threaded View
Subject: Re: Reg Kafka Replication
Hi Balasubramanian
Do you mean, why does Broker 0 stay in the replication set for partitions when it goes down?

This seems like a design decision as it means if 0 has a transient failure, due to a network blip or a restart, it will rejoin it's partitions when it comes back up. This would result in the least network activity as it would have most of the log already on disk and would just need to catch up.

One could also imagine a scenario where a broker went down because of load. This broker's partition assignments would get reassigned to other highly loaded brokers which could cause a cascading failure (I think).

You can reassign partitions with It is also a very good idea to monitor the JMX bean for under replicated partitions as this would indicate the partitions at risk.

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB