Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # dev - Review Request 16175: Patch for KAFKA-1178


Copy link to this message
-
Re: Review Request 16175: Patch for KAFKA-1178
Neha Narkhede 2013-12-11, 18:10


> On Dec. 11, 2013, 5:08 p.m., Guozhang Wang wrote:
> > core/src/main/scala/kafka/server/ReplicaManager.scala, line 237
> > <https://reviews.apache.org/r/16175/diff/1/?file=396497#file396497line237>
> >
> >     Under which condition the broker will not be in the assigned replica list? Since the controller have full control of sending LeaderAndIsr request to brokers, this case sounds more like a bug to me.

During partition reassignment, when new set of replicas are communicated, the request is sent to all leaders and followers. But it can happen that during the shrinking of the replication factor, a follower is not in the assigned replica list (for a small period of time). This protects against that case
- Neha
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16175/#review30202
-----------------------------------------------------------
On Dec. 11, 2013, 6:30 a.m., Neha Narkhede wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16175/
> -----------------------------------------------------------
>
> (Updated Dec. 11, 2013, 6:30 a.m.)
>
>
> Review request for kafka.
>
>
> Bugs: KAFKA-1178
>     https://issues.apache.org/jira/browse/KAFKA-1178
>
>
> Repository: kafka
>
>
> Description
> -------
>
> 1. Fixed broker to only follow become-follower/leader request if the broker exists in the assigned replica list 2. Changed the under replicated check to compare in sync replica set size with assigned replica set size instead of passed in replication factor 3. Fixed the fetcher id computation to not create more fetcher threads than necessary due to the negative (overflown) hashcode value. 4. Fixed some more state change logging 5. Tested partition reassignment several times on a cluster with several 100s of topics, verified reassignment completion followed by no under replicated partitions (through jmx and through zookeeper)
>
>
> Diffs
> -----
>
>   core/src/main/scala/kafka/admin/ReassignPartitionsCommand.scala 65c04edc11a1eb2771bfd741d59886b5fef9ccbc
>   core/src/main/scala/kafka/cluster/Partition.scala 13f48baa1bc44533c33153fde6f1c03a067d83bc
>   core/src/main/scala/kafka/server/AbstractFetcherManager.scala 394e981c858fa53a28d3c7fcef0afbd4faf07858
>   core/src/main/scala/kafka/server/ReplicaManager.scala b0a39622ab91e209f9657231917a5ae30fd835f0
>
> Diff: https://reviews.apache.org/r/16175/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Neha Narkhede
>
>