Kafka, mail # user - Too many replicas after partition reassignment - 2014-04-21, 18:41
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Too many replicas after partition reassignment
After doing some partition reassignments, I've ended up with some
partitions that have both the old and new brokers assigned.

The output of kafka-topics.sh --describe looks like this:

Topic:cs-es-indexer-a PartitionCount:30 ReplicationFactor:2 Configs:
retention.ms=10800000
...
  Topic: cs-es-indexer-a  Partition: 11 Leader: 11  Replicas: 11,12 Isr:
11,12
  Topic: cs-es-indexer-a  Partition: 12 Leader: 15  Replicas: 15,17,12
 Isr: 12,17,15
...

Partition 12 has 3 replicas, but the topic has a replication factor of 2.
It was reassigned from [12,17] to [15,17].

During the reassignment, the controller changed from one broker to another,
and the resigning controller deadlocked with
https://issues.apache.org/jira/browse/KAFKA-1317.

Is this expected, or a known issue? I've had to alter some of my monitoring
tools to handle different-sized replica lists for partitions of the same
topic.

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB