Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # dev >> [jira] [Commented] (KAFKA-782) replicas not being displayed in TopicMetadataResponse when replica's Broker is shutdown


Copy link to this message
-
[jira] [Commented] (KAFKA-782) replicas not being displayed in TopicMetadataResponse when replica's Broker is shutdown

    [ https://issues.apache.org/jira/browse/KAFKA-782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13645744#comment-13645744 ]

Neha Narkhede commented on KAFKA-782:
-------------------------------------

This is probably fixed by now, can you retry ?
                
> replicas not being displayed in TopicMetadataResponse when replica's Broker is shutdown
> ---------------------------------------------------------------------------------------
>
>                 Key: KAFKA-782
>                 URL: https://issues.apache.org/jira/browse/KAFKA-782
>             Project: Kafka
>          Issue Type: Bug
>          Components: replication
>    Affects Versions: 0.8
>         Environment: $ uname -a
> Linux vrd01.atlnp1 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
> $ java -version
> java version "1.6.0_25"
> Java(TM) SE Runtime Environment (build 1.6.0_25-b06)
> Java HotSpot(TM) 64-Bit Server VM (build 20.0-b11, mixed mode)
> Kafka 0.8.0 loaded from HEAD on 1/29/2013
>            Reporter: Chris Curtin
>            Assignee: Neha Narkhede
>              Labels: kafka-0.8, p2
>
> Setup
> - 4 nodes in the cluster
> - topic has 10 partitions, numbered 0-9 in the output below
> - configure the cluster so one of the nodes doesn't hold any leader copies of the partition. I did this by shutting down a node, waiting for reassignment of leaders and starting again
> Output of the TopicMetadataResponse call:
> Partition: 0:vrd01.atlnp1 R:[  vrd03.atlnp1 vrd04.atlnp1 vrd01.atlnp1] I:[ vrd01.atlnp1 vrd03.atlnp1 vrd04.atlnp1]
> Partition: 1:vrd01.atlnp1 R:[  vrd04.atlnp1 vrd01.atlnp1 vrd02.atlnp1] I:[ vrd01.atlnp1 vrd04.atlnp1 vrd02.atlnp1]
> Partition: 2:vrd01.atlnp1 R:[  vrd01.atlnp1 vrd02.atlnp1 vrd03.atlnp1] I:[ vrd01.atlnp1 vrd03.atlnp1 vrd02.atlnp1]
> Partition: 3:vrd03.atlnp1 R:[  vrd02.atlnp1 vrd03.atlnp1 vrd04.atlnp1] I:[ vrd03.atlnp1 vrd04.atlnp1 vrd02.atlnp1]
> Partition: 4:vrd01.atlnp1 R:[  vrd03.atlnp1 vrd01.atlnp1 vrd02.atlnp1] I:[ vrd01.atlnp1 vrd03.atlnp1 vrd02.atlnp1]
> Partition: 5:vrd03.atlnp1 R:[  vrd04.atlnp1 vrd02.atlnp1 vrd03.atlnp1] I:[ vrd03.atlnp1 vrd04.atlnp1 vrd02.atlnp1]
> Partition: 6:vrd01.atlnp1 R:[  vrd01.atlnp1 vrd03.atlnp1 vrd04.atlnp1] I:[ vrd01.atlnp1 vrd03.atlnp1 vrd04.atlnp1]
> Partition: 7:vrd01.atlnp1 R:[  vrd02.atlnp1 vrd04.atlnp1 vrd01.atlnp1] I:[ vrd01.atlnp1 vrd04.atlnp1 vrd02.atlnp1]
> Partition: 8:vrd03.atlnp1 R:[  vrd03.atlnp1 vrd02.atlnp1 vrd04.atlnp1] I:[ vrd03.atlnp1 vrd04.atlnp1 vrd02.atlnp1]
> Partition: 9:vrd01.atlnp1 R:[  vrd04.atlnp1 vrd03.atlnp1 vrd01.atlnp1] I:[ vrd01.atlnp1 vrd03.atlnp1 vrd04.atlnp1]
> Note that vrd02.atlnp1 does not contain any leader replicas.
> Shutdown vrd02 normally. Run TopicMetadataResponse again:
> Partition: 0:vrd01.atlnp1 R:[  vrd03.atlnp1 vrd04.atlnp1 vrd01.atlnp1] I:[ vrd01.atlnp1 vrd03.atlnp1 vrd04.atlnp1]
> Partition: 1:vrd01.atlnp1 R:[ ] I:[]
> Partition: 2:vrd01.atlnp1 R:[ ] I:[]
> Partition: 3:vrd03.atlnp1 R:[ ] I:[]
> Partition: 4:vrd01.atlnp1 R:[ ] I:[]
> Partition: 5:vrd03.atlnp1 R:[ ] I:[]
> Partition: 6:vrd01.atlnp1 R:[  vrd01.atlnp1 vrd03.atlnp1 vrd04.atlnp1] I:[ vrd01.atlnp1 vrd03.atlnp1 vrd04.atlnp1]
> Partition: 7:vrd01.atlnp1 R:[ ] I:[]
> Partition: 8:vrd03.atlnp1 R:[ ] I:[]
> Partition: 9:vrd01.atlnp1 R:[  vrd04.atlnp1 vrd03.atlnp1 vrd01.atlnp1] I:[ vrd01.atlnp1 vrd03.atlnp1 vrd04.atlnp1]
> Note that the partitions where vrd02 was in the replica set no longer show any replicas.
> Not clear if the list of replicas isn't being set correctly or the replicas aren't associated with the partition any longer.
> Java code:
>   kafka.javaapi.consumer.SimpleConsumer consumer  = new SimpleConsumer("vrd01.atlnp1",
>                 9092,
>                 100000,
>                 64 * 1024, "test");
>         List<String> topics2 = new ArrayList<String>();
>         topics2.add("storm-anon");
>         TopicMetadataRequest req = new TopicMetadataRequest(topics2);

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira