Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # dev >> [jira] [Commented] (KAFKA-901) Kafka server can become unavailable if clients send several metadata requests


+
Joel Koshy 2013-05-10, 17:57
Copy link to this message
-
[jira] [Commented] (KAFKA-901) Kafka server can become unavailable if clients send several metadata requests

    [ https://issues.apache.org/jira/browse/KAFKA-901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13659005#comment-13659005 ]

Neha Narkhede commented on KAFKA-901:
-------------------------------------

That's correct.
                
> Kafka server can become unavailable if clients send several metadata requests
> -----------------------------------------------------------------------------
>
>                 Key: KAFKA-901
>                 URL: https://issues.apache.org/jira/browse/KAFKA-901
>             Project: Kafka
>          Issue Type: Bug
>          Components: replication
>    Affects Versions: 0.8
>            Reporter: Neha Narkhede
>            Assignee: Neha Narkhede
>            Priority: Blocker
>         Attachments: kafka-901.patch, metadata-request-improvement.patch
>
>
> Currently, if a broker is bounced without controlled shutdown and there are several clients talking to the Kafka cluster, each of the clients realize the unavailability of leaders for some partitions. This leads to several metadata requests sent to the Kafka brokers. Since metadata requests are pretty slow, all the I/O threads quickly become busy serving the metadata requests. This leads to a full request queue, that stalls handling of finished responses since the same network thread handles requests as well as responses. In this situation, clients timeout on metadata requests and send more metadata requests. This quickly makes the Kafka cluster unavailable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

 
+
Jun Rao 2013-05-17, 16:29
+
Neha Narkhede 2013-05-21, 17:09
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB